Search

Due to the emergence and advancement of deep learning technologies, scene text detection is becoming more widespread in various fields. However, due to the complexity of distances, angles and backgrounds, the adjacent texts in images have the problem that the detection boxes are far away from the texts, i.e., a position is not accurate enough. In this paper, we propose a text detection method centered on double ResNet-based and changed channels recursive feature pyramid, which integrates ResNet50-Mish and Res2Net50-Mish, as well as using recursive feature pyramid with changed channels. Firstly, scene images are fed into ResNet50-Mish and Res2Net50-Mish of double ResNet-based, and results are passed through a weight-based addition step to generate the fused feature maps. Secondly, the processed feature maps of double ResNet-based are sent into changed channels recursive feature pyramid to obtain feature maps with enhanced feature information. Also, the relevant segmentation results are then obtained by concatenating and convoluting. Finally, the results are given to progressive scale expansion algorithm to output the location of texts in images. The proposed model is trained and tested on ICDAR15 and CTW1500 benchmark datasets. In terms of precision values, our method outperforms or is comparable to state-of-the-art methods. In particular, experimental results achieve 91.53% precision on ICDAR15 dataset and 84.89% precision on CTW-1500 dataset. [ABSTRACT FROM AUTHOR]

In recent years, deep metric learning and its probabilistic extensions claimed state-of-the-art results in the face verification task. Despite improvements in face verification, probabilistic methods received little attention in the research community and practical applications. Previous metric learning benchmarks avoided comparison with probabilistic methods, and it is still unclear whether they generalize well to image retrieval tasks beyond faces. In this paper, we, for the first time, perform an in-depth analysis and unified comparison of known probabilistic methods in verification and retrieval tasks. We study different design choices and argue that many of them have limited impact, while only a few probabilistic methods surpass modern metric learning approaches. Combining the best ideas of earlier works, we propose a simple modification of the top-performing method, achieving new state-of-the-art results among probabilistic methods. Finally, we study confidence prediction and show that it correlates with data quality, but contains little information about prediction error probability. We thus provide a new confidence evaluation benchmark and establish a baseline for future confidence prediction research. PyTorch implementation is publicly released. [ABSTRACT FROM AUTHOR]

3D point cloud registration is a basic task in computer vision. In recent years, various of learning-based methods have been proposed to solve this problem. These methods effectively overcome the original problem of over-reliance on initial conditions and enhance the ability of obtaining the corresponding relationship. However, few methods pay enough attention to local features and tend to cause some mismatches. Therefore, this paper proposes two networks to extract local features sufficiently. To obtain a more accurate correspondence relationship between the point clouds, we propose a feature weight allocation network (FWANet), in which the expression ability of feature is enhanced using the proposed significant feature extraction module. Besides that, we utilize an interference elimination module to remove the interference points and enhance the internal correlation of point clouds. We also propose a spatial structural generation network (SSGNet), which fully utilizes the spatial location information to determine the spatial correspondence and generate a reliable connection after concatenating multi-dimensional features. At last, a complete feature space can be effectively captured after combining our FWANet with SSGNet together. We conducted multiple experiments on ModelNet40 datasets and achieved excellent results. Experimental results on four types of data demonstrate the superiority of our algorithm against the state-of-the-art ones. Our code will be available at https://github.com/liu-zikang/registration as soon as the paper is accepted. [ABSTRACT FROM AUTHOR]

Accurately localizing bare-hands is crucial for human–computer interaction systems. In real-time systems, computational time plays a pivotal role in achieving this task within a specified timeframe. The localization process may involve detection, tracking, or both, depending on the framework's needs. Most studies on bare-hand detection and tracking have been conducted in controlled environments. However, localizing bare-hands in uncontrolled environments are challenging due to the complexity of variations, such as changes in illumination, rotation, occlusion, scale, pose, speed, and impostor bare-hands. These factors can significantly impact the performance of the models, along with background feature domination effect and motion blur, which further complicate localization. To address these challenges, this paper presents a comprehensive analysis of the most significant deep learning-based hand localization models. We have customized an object detector as a bare-hand localization model by incorporating detection and tracking modules, providing computationally efficient performance while addressing the variations. To evaluate the models' efficiency across a range of scales, we have proposed a scale-based bare-hand database with varying scales from 20 to 200 cm. We have also evaluated these models on various bare-hand benchmark databases. [ABSTRACT FROM AUTHOR]

With the gradual growth of deep learning in machine vision, efficient extraction of 3D point clouds becomes significant. The raw data of the 3D point cloud are sparse, disordered, and immersed in noise, which makes it difficult to classify and segment. Whether 3D point clouds can be classified and segmented or not, the local feature is an essential ingredient. Therefore, this paper proposes a GateNet-based PointNet++ network (G-PointNet++). G-PointNet++ extracts local features more accurately than PointNet++ by suppressing irrelevant features and emphasizing important features. Meanwhile, it refines the feature adaptively. Besides, the SENet and attention mechanism are introduced into PointNet++. G-PointNet++ was evaluated on the public ModelNet dataset, ShapeNet dataset, and S3DIS dataset, and its effectiveness in classification and segmentation tasks was verified. In the classification task, G-PointNet++ achieves an overall classification accuracy (OA) of 95.5% on ModelNet10 and 93.3% on ModelNet40. In the segmentation task, the mIoU of G-PointNet++ reaches 85.5% on ShapeNet. These experiments show that G-PointNet++ achieves better performance and saves more time than PointNet++, and its overall accuracy is higher than that of PCT network on ModeNet40. [ABSTRACT FROM AUTHOR]

Rice which is a staple food crop in most Asian countries mainly suffers from higher yield loss due to different factors, and one of the common factors that affect rice yield is rice leaf disease. Many countries do not allow the plantation of new rice varieties unless they are disease-resistant. Hence, disease control serves as an important factor for rice production and this can be effectively accomplished by identifying the rice disease at an early stage which is required for timely pesticide application and disease control. The rice leaf diseases are mainly inspected by the farmers with their naked eye which is error-prone, time-consuming, and often leads to huge yield loss if predicted wrong. This process is effectively handled by different researchers via the use of computer vision and machine learning techniques. To overcome the drawbacks associated with manual processing and low recognition accuracy of different statistical and machine learning techniques, this paper presents a novel concept that uses 3D 2D CNN for feature extraction and an improved backtracking search (IBS) algorithm-optimized deep generative adversarial network (DGAN) for classification. The 3D2D deep convolutional neural network (DCNN) is formed by integrating the 3D fast-learning block with a 2DCNN for extracting the rice disease features such as lesions and shape. The IBS algorithm-optimized GAN architecture is used for rice disease classification and the IBS algorithm is mainly adopted to overcome the instability and overfitting issues that arise in the DGAN. The proposed IBS-optimized GAN architecture offers improved accuracy of 98.7% which is relatively higher than the existing techniques such as extreme gradient boosting (XGBoost), transfer learning techniques, and support vector machine (SVM). [ABSTRACT FROM AUTHOR]

Automatic facial expression recognition plays a crucial role in computer vision, and pattern recognition. However, most existing deep learning-based facial expression classifiers usually obtain high average accuracy but have poor recognition accuracy for difficult expressions, like fear and disgust. In this paper, we propose a novel end-to-end architecture termed two-stream inter-class variation enhancement network, which learns the high-level semantic features and subtle inter-class variations in a joint fashion. More precisely, the global feature extraction network is used to extract spatial-channel semantic features, and the variations between different expressions are modeled by a distinction-reinforced network. The outputs of these two streams are weighted integrated in the expression classification network. In addition, a class balanced-weighted cross-entropy loss is designed to further improve feature discrimination. Experiment results indicate that the proposed network can significantly improve the recognition of difficult expressions and achieve a satisfactory average recognition accuracy of 73.67% on FER2013, 86.17% on RAFDB, 98.19% on CK+, and 98.85% on Oulu-CASIA, which outperforms the other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

3D point set reconstruction is an important and challenging 3D shape analysis task. Current state-of-the-art algorithms for 3D point set reconstruction employ a deep neural network (DNN) having an encoder–decoder architecture. Recently, the decoder DNNs that transform multiple 2D planar patches to reconstruct a 3D shape have seen some success. These "patch-folding" decoders are adept at approximating smooth surfaces in 3D objects. However, 3D point sets generated by these decoders often lack local geometrical details, as 2D planar patches tend to overly constrain the patch folding process. In this paper, we propose a novel decoder DNN for 3D point sets called Hyperplane Mixing and Folding Net (HMF-Net). HMF-Net uses less constrained hyperplane, not 2D plane, patches as its input to the folding process. HMF-Net has, as its core building block, a stack of token-mixing layers to effectively learn global consistency among the hyperplane patches. In addition to HMF-Net, we also propose a novel loss for 3D point set reconstruction called Weighted Chamfer Distance (WCD). WCD tries to weight, or amplify, loss from parts of shape that are highly variable across training samples by emphasizing higher point-pair distance values between a generated point set and a groundtruth point set. This helps the decoder DNN learn shape details better. We comprehensively evaluate our algorithm under three 3D point set reconstruction scenarios, that are, shape completion, shape upsampling, and shape reconstruction from 2D images. Experimental results demonstrate that our algorithm yields accuracies higher than the existing algorithms for 3D point set reconstruction. [ABSTRACT FROM AUTHOR]

Pose-invariant facial expression recognition is one of the popular research directions within the field of computer vision, but pose variant usually change the facial appearance significantly, making the recognition results unstable from different perspectives. In this paper, a novel deep learning method, namely, soft thresholding squeeze-and-excitation (ST-SE) block, was proposed to extract salient features of different channels for pose-invariant FER. For the purpose of adapting to different pose-invariant facial images better, global average pooling (GAP) operation was adopted to compute the average value of each channel of the feature map. To enhance the representational power of the network, Squeeze-and-Excitation (SE) block was embedded into the nonlinear transformation layer to filter out the redundant feature information. To further shrink the significant features, the absolute values of GAP and SE were multiplied to calculate the threshold suitable for the current view. And the developed ST-SE block was inserted into ResNet50 for the evaluation of recognition performance. In this study, extensive experiments on four pose-invariant datasets were carried out, i.e., BU-3DFE, Multi-PIE, Pose-RAF-DB and Pose-AffectNet, and the influences of different environments, poses and intensities on expression recognition were specifically analyzed. The experimental results demonstrate the feasibility and effectiveness of our method. [ABSTRACT FROM AUTHOR]

Chest radiography (X-ray) is the most common diagnostic method for pulmonary disorders. A trained radiologist is required for interpreting the radiographs. But sometimes, even experienced radiologists can misinterpret the findings. This leads to the need for computer-aided detection diagnosis. For decades, researchers were automatically detecting pulmonary disorders using the traditional computer vision (CV) methods. Now the availability of large annotated datasets and computing hardware has made it possible for deep learning to dominate the area. It is now the modus operandi for feature extraction, segmentation, detection, and classification tasks in medical imaging analysis. This paper focuses on the research conducted using chest X-rays for the lung segmentation and detection/classification of pulmonary disorders on publicly available datasets. The studies performed using the Generative Adversarial Network (GAN) models for segmentation and classification on chest X-rays are also included in this study. GAN has gained the interest of the CV community as it can help with medical data scarcity. In this study, we have also included the research conducted before the popularity of deep learning models to have a clear picture of the field. Many surveys have been published, but none of them is dedicated to chest X-rays. This study will help the readers to know about the existing techniques, approaches, and their significance. [ABSTRACT FROM AUTHOR]

Moving object detection in low-luminance Images is one of the most fundamental and difficult issues in machine vision. Therefore, in this paper, deep self-adaptive network (DSA-Net) is proposed to effectively detect moving objects in low-luminance scenes. Particularly, (1) three mechanisms are developed in this joint learning framework: graph-based unsupervised feature selection, feature representations ranking, and multiple-way feature interaction. (2) Both anti-occlusion and multi-object handling module are explored simultaneously in the unified DSA-Net model. (3) A weakly fine-tuning strategy is presented, including the easiness and group curriculum term. It leverages helpful prior-knowledge to guide the learner to select confident training samples. The experimental results show that DSA-Net outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Underwater image enhancement (UIE), as an image processing technique, plays a vital role in computer vision. However, existing approaches treat the restoration process as a whole; thus, they cannot adequately handle the color distortion and low contrast in the enhanced images. In this paper, we propose a global–local-guided model for realizing UIE tasks in a coarse-to-fine manner to alleviate these issues. The proposed model is divided into two paths. The global path targets to estimate basic structure and color information, while the local path targets to remove the undesirable artifacts, e.g., noises over-exposure regions, and blurred edges. By integrating two neural networks into our model, we could recover the underwater images with clear textural details and vivid color. Besides, a learning-based weight map is introduced to make the global–local path on friendly terms, which can balance the pixel intensity distribution from both sides and remove redundant information to a certain degree. Qualitative and quantitative experimental results on various benchmarks demonstrate that our method can effectively tackle color distortion and blurred edges compared with several state-of-the-art methods by a large margin. Finally, we also conduct experiments to demonstrate that our method can be applied in various computer vision tasks, e.g., object detection, matching and edge detection. [ABSTRACT FROM AUTHOR]

Single image rain streak removal is a well-explored topic in the field of computer vision. The de-raining problem is modeled as an image decomposition task where a rainy image is decomposed into rain-free background image and rain streek map. Unlike most of the existing de-raining methods, this paper attempts to decompose the rainy image in the frequency domain. The idea is inspired by pseudo-periodic characteristics of the noise signal (here the rain streaks) which leave some traces in the frequency domain, and the same can be utilized to predict the noise signal. In this paper, a deep learning-based rain streak prediction model is proposed which learns in discrete Fourier transform Oppenheim and Schafer (Discrete-TimeSignal Processing, Prentice Hall, Upper Saddle River, 1989) domain. To the best of our knowledge, this is the first approach where compressed domain coefficients are directly used as input to a deep convolutional neural network. The proposed model has been tested on publicly available synthetic datasets Fu et al. (in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. https://doi.org/10.1109/CVPR.2017.186, Yang et al. (in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. https://doi.org/10.1109/CVPR.2017.183), Yeh et al. (in: 2015 IEEE International Conference on Consumer Electronics-Taiwan, 2015. https://doi.org/10.1109/ICCE-TW.2015.7216999) and results are found to be comparable with the state of the art methods in the spatial domain. The presented analysis and study have an obvious indication to extend transform domain input to train the deep learning architecture especially image de-noising like problems. [ABSTRACT FROM AUTHOR]

Visual object tracking has become one of the most active research topics in computer vision, which has been growing in commercial development as well as academic research. Many visual trackers have been proposed in the last two decades. Recent studies of computer vision for dynamic scenes include motion detection, object classification, environment modeling, tracking of moving objects, understanding of object behaviors, object identification, and data fusion from multiple sensors. This paper provides an in-depth overview of recent object tracking research. Object tracking tasks in realistic scenario often face challenging problems such as camera motion, occlusion, illumination effect, clutter, and similar appearance. A variety of tracker techniques have been published, which combine multiple techniques to solve multiple visual tracking sub-problems. This paper also reviews the latest research trend in object tracking based on convolutional neural networks, which is receiving growing attention. Finally, the paper discusses the future challenges and research directions for the object tracking problems that still need extensive studies in coming years. [ABSTRACT FROM AUTHOR]

The research progress in multimodal learning has grown rapidly over the last decade in several areas, especially in computer vision. The growing potential of multimodal data streams and deep learning algorithms has contributed to the increasing universality of deep multimodal learning. This involves the development of models capable of processing and analyzing the multimodal information uniformly. Unstructured real-world data can inherently take many forms, also known as modalities, often including visual and textual content. Extracting relevant patterns from this kind of data is still a motivating goal for researchers in deep learning. In this paper, we seek to improve the understanding of key concepts and algorithms of deep multimodal learning for the computer vision community by exploring how to generate deep models that consider the integration and combination of heterogeneous visual cues across sensory modalities. In particular, we summarize six perspectives from the current literature on deep multimodal learning, namely: multimodal data representation, multimodal fusion (i.e., both traditional and deep learning-based schemes), multitask learning, multimodal alignment, multimodal transfer learning, and zero-shot learning. We also survey current multimodal applications and present a collection of benchmark datasets for solving problems in various vision domains. Finally, we highlight the limitations and challenges of deep multimodal learning and provide insights and directions for future research. [ABSTRACT FROM AUTHOR]

Recently, the world has witnessed a great increase in drone applications and missions. Drones must be detected quickly, effectively, and precisely when they are being handled illegally. Vision-based anti-drone systems provide an efficient performance compared to radar- and acoustic-based systems. The effectiveness of drone detection is affected by a number of issues, including the drone’s small size, conflicts with other objects, and noisy backgrounds. This paper employs enlarging the effective receptive field (ERF) of feature maps generated from the YOLOv6 backbone. First, RepLKNet is used as the backbone of YOLOv6, which deploys large kernels with depth-wise convolution. Then, to get beyond RepLKNet’s large inference time, a novel LERFNet is implemented. LERFNet uses dilated convolution in addition to large kernels to enlarge the ERF and overcome each other’s problems. The linear spatial-channel attention module (LAM) is used to give more attention to the most informative pixels and high feature channels. LERFNet produces output feature maps with a large ERF and high shape bias to enhance the detection of various drone sizes in complex scenes. The RepLKNet and LERFNet backbones for Tiny-YOLOv6, Tiny-YOLOv6, YOLOv5s, and Tiny-YOLOv7 are compared. In comparison to the aforementioned techniques, the suggested model’s results show a greater balance between accuracy and speed. LERFNet increases the MAP by 2.8%\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$2.8\%$$\end{document}, while significantly reducing the GFLOPs and parameter numbers when compared to the original backbone of YOLOv6. [ABSTRACT FROM AUTHOR]

Manual harvesting of loose fruits in the oil palm plantation is both time consuming and physically laborious. Automatic harvesting system is an alternative solution for precision agriculture which requires accurate visual information of the targets. Current state-of-the-art one-stage object detection method provides excellent detection accuracy; however, it is computationally intensive and impractical for embedded system. This paper proposed an improved YOLO model to detect oil palm loose fruits from unmanned aerial vehicle images. In order to improve the robustness of the detection system, the images are augmented by brightness, rotation, and blurring to simulate the actual natural environment. The proposed improved YOLO model adopted several improvements; densely connected neural network for better feature reuse, swish activation function, multi-layer detection to enhance detection on small targets and prior box optimization to obtain accurate bounding box information. The experimental results show that the proposed model achieves outstanding average precision of 99.76% with detection time of 34.06 ms. In addition, the proposed model is also light in weight size and requires less training time which is significant in reducing the hardware costs. The results exhibit the superiority of the proposed improved YOLO model over several existing state-of-the-art detection models. [ABSTRACT FROM AUTHOR]

Intelligent video surveillance systems have been used recently for automatic monitoring of human interactions. Although they play a significant role in reducing security concerns, there are many challenges for distinguishing between normal and abnormal behaviors such as crowded environments and camera viewpoint. In this paper, we propose a novel deep violence detection framework based on the specific features derived from handcrafted methods. These features are related to appearance, speed of movement, and representative image and fed to a convolutional neural network (CNN) as spatial, temporal, and spatiotemporal streams. The spatial stream trained the network with each frame in the video to learn environment patterns. The temporal stream contained three consecutive frames to learn motion patterns of violent behavior with a modified differential magnitude of optical flow. Moreover, in spatio-temporal stream, we introduced a discriminative feature with a novel differential motion energy image to represent violent actions more interpretable. This approach covers different aspects of violent behavior by fusing the results of these streams. The proposed CNN network is trained with violence-labeled and normal-labeled frames of 3 Hockey, Movie, and ViF datasets which comprised both crowded and uncrowded situations. The experimental results showed that the proposed deep violence detection approach outperformed state-of-the-art works in terms of accuracy and processing time. [ABSTRACT FROM AUTHOR]

Human skin detection, which is one of the important pre-processing phases, has a wide range of applications such as face tracking, skin diseases, video surveillance, web content filtering, and so on. Skin detection is a challenging problem because skin color can vary dramatically in its appearance due to many factors such as illumination conditions, pose variations, race, aging, and complex background. Several methods dealing with skin detection assume that skin pixels can be extracted from background colors according to some thresholding rules related to a specific color model. Nevertheless, it is a complex task to recognize skin pixels under the challenging factors aforementioned. In the recent era, the success of deep convolutional neural network (CNN) has strongly influenced the field of computer vision. However, we could find only a few researches that apply deep learning methods to deal with the skin detection problem. This paper presents a novel approach based on CNN for skin detection. Extensive experiments show that the proposed approach exceeds the best result for other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In view of the multi-scale changes and the influence of light and angle in the image matching process, it is quite difficult to realize intelligent image registration by using convolutional neural network. The existing image matching algorithm has the following problems in the application process: the existing shallow feature extraction model has lost a lot of effective feature information and low recognition accuracy. Meanwhile, the image registration method based on deep learning is not robust and accurate enough. Therefore, an image registration method based on additive edge cosine loss was proposed in this paper. In the twin network architecture, cosine loss was used to convert Euclidean space into angular space, which eliminated the influence of characteristic intensity and improved the accuracy of registration. The matching cost was directly calculated by the included angle of two vectors in the embedded space, where the size of the angle edge could be quantitatively adjusted through parameter m . We further derived a specific m to quantitatively adjust the loss. In addition, anti-rotation attention mechanism was added to the network to enhance the ability of feature information extraction and adjust the position information of feature vectors to reduce the mismatching caused by image rotation. [ABSTRACT FROM AUTHOR]

The experimental purpose of X-ray diffraction is to analyze crystalline material structure at the atomic and molecular levels. Such experiments are known as X-ray crystallography. Traditionally, human experts do it with some domain knowledge. X-ray crystallography is useful in numerous domains, e.g., physics, chemistry, and biology. It is tough to own manual physics of diffraction patterns to see a crystal structure with a colossal data set. A convolutional neural network (CNN) is a deep neural network that maps an input image into a high-dimensional space. CNN produces an affordable function for image classification. This paper uses an extension of the convolutional neural network to predict crystal structure from diffraction patterns. We propose a machine-enabled method to predict crystallographic size and space group from a limited number of XRD patterns for small films. We overcome the problem of scarce data within the development of building materials by combining the learning model of moderately monitored equipment, a physics information-enhancing strategy using data generated from the Inorganic Crystal Structure Database, and test data. We compare our approach with a large variety of typical addition as modern machine learning-based classification techniques for crystal structure prediction. Results show that our proposed system outperforms all the baselines by a significant margin for the crystal structure prediction task. Results also show the impact of the number of layers in the all-convolutional neural network architecture for crystal structure prediction. [ABSTRACT FROM AUTHOR]

Computer vision is becoming an increasingly critical area of research, and its applications to real-world problems are gaining significance. In this paper, we describe the design, development and evaluation of our computer vision Faster R-CNN iPad App for Sales Representatives in grocery store environments. Our system aims to assist Sales Reps to be more productive, reduce errors, and provide increased efficiencies. We report on the creation of the iPad app, the data capturing guidelines we created for the creation of good classifiers and the results of professional Sales Reps evaluating our system. Our system was tested in a variety of conditions in grocery store environments and has an accuracy of 99%, a System Usability Score usability score of 85 (high). It supports up to 40 classifiers running concurrently to perform product identification in less than 3.8 s. We also created a set of data capturing guidelines that will enable other researchers to create their own classifiers for these types of products in complex environments (e.g., products with very similar packaging located on shelves). [ABSTRACT FROM AUTHOR]

Video frame interpolation is an important area in the computer vision research activities for video post-processing, surveillance, and video restoration tasks. It aims toward increasing the frame rate of a video sequence by calculating intermittent frames between consecutive input frames. This ensures extra smooth, clear motion in order to make animation fluid enough and reduce display motion blur. Advanced deep learning algorithms have the potential to discover knowledge from large-scale diverse video data. These algorithms gain insights about intermediate motion and provide new opportunities to further improve video interpolation technologies. This survey demonstrates a comprehensive overview of about a good number of contributions over past decade pertinent to the latest developments in this domain. The survey paper highlights common challenges in the area of video frame interpolation based on three key aspects: high visual quality, low complexity, and high efficiency of interpolated output from regular videos with the standard frame rate. We scrutinize the architectures, workflows, performance, advantages, and disadvantages and generate a broad categorization along with an overview of experimental results of various state-of-the-art methods executed on benchmark datasets. This survey discusses applications of diverse interpolation frameworks. It provides a backbone reference that inspires future researchers to optimize current techniques on academic and industrial grounds. [ABSTRACT FROM AUTHOR]

The presence of fog degrades visibility in natural scene conditions. Computer vision applications like navigation, tracking, and surveillance need clear atmospheric images or videos as prerequisites for optimal performance. However, foggy atmosphere creates problems for computer vision applications due to reduced visibility. Different fog removal techniques are used to improve the visual quality of images and videos. The fog density depends on the depth information. Scene depth information estimation needs multiple images, which limits its real-life application. Hence, a single image fog removal requires some prior knowledge and/or assumptions to get the depth information. In this paper, the recent fog removal techniques are grouped into three broad categories: (1) filter-based methods, (2) color correction based methods, and (3) learning-based methods, for ease of understanding. The primary objective is to provide an introduction to this field and compare performance (both qualitative and quantitative) of representative techniques for each category. It is found that filter-based methods are doing overall better compared to other categories. [ABSTRACT FROM AUTHOR]

Human action recognition is still a challenging topic in the computer vision field that has attracted a large number of researchers. It has a significant importance in varieties of applications such as intelligent video surveillance, sports analysis, and human–computer interaction. Recent works attempt to exploit the progress in deep learning architecture to learn spatial and temporal features from action video. However, it remains unclear how to combine spatial and temporal information with convolutional neural network. In this paper, we propose a novel human action recognition method by fusing spatial and temporal features learned from a simple unsupervised convolutional neural network called principal component analysis network (PCANet) in combination with bag-of-features (BoF) and vector of locally aggregated descriptors (VLAD) encoding schemes. Firstly, both spatial and temporal features are learned via PCANet using a subset of frames and temporal templates for each video, while their dimensionality is reduced using whitening transformation (WT). The temporal templates are calculated using short-time motion energy images (ST-MEI) based on frame differencing. Then, the encoding scheme is applied to represent the final dual spatiotemporal PCANet features by feature fusion. Finally, the support vector machine (SVM) classifier is exploited for action recognition. Extensive experiments have been performed on two popular datasets, namely KTH and UCF sports, to evaluate the performance of proposed method. Our experimental results using leave-one-out evaluation strategy demonstrate that the proposed method presents satisfactory and comparable results on both datasets. [ABSTRACT FROM AUTHOR]

With the advent of deep learning, multi-modal data have been of great interest. One of the multi-modal tasks which can be included in the computer vision domain is visual question answering (VQA). In VQA, a question and an image are entered into the model and the model tries to answer the question according to the image. To the best of our knowledge, the current techniques look at the image and only give one answer to the question asked. However, in some situations, there are several answers to the asked question. In this paper, we address this problem and define a new domain in the task of VQA as well as a new computationally efficient approach to cope with multiple-answer VQA. In this approach, we use a sliding window in an efficient manner to examine the answer to the question in different parts of the image. Due to the fact that so far no proper dataset is available for multiple-answer VQA, we provide a new dataset for evaluating our proposed model. The experiments express that our model uses 94% less operation than other models, making it very suitable for real-time applications. [ABSTRACT FROM AUTHOR]

Single image dehazing, which is the process of removing haze from a single input image, is an important task in computer vision. This task is extremely challenging because it is massively ill-posed. In this paper, we propose a novel end-to-end deep residual convolutional dehazing network (DRCDN) based on convolutional neural networks for single image dehazing, which consists of two subnetworks: one network is used for recovering a coarse clear image, and the other network is used to refine the result. The DRCDN firstly predicts the coarse clear image via a context aggregation subnetwork, which can capture global structure information. Subsequently, it adopts a novel hierarchical convolutional neural network to further refine the details of the clean image by integrating the local context information. The DRCDN is directly trained using complete images and the corresponding ground-truth haze-free images. Experimental results on synthetic datasets and natural hazy images demonstrate that the proposed method performs favorably against the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Interest in automatic crowd behaviour analysis has grown considerably in the last few years. Crowd behaviour analysis has become an integral part all over the world for ensuring peaceful event organizations and minimum casualties in the places of public and religious interests. Traditionally, the area of crowd analysis was computed using handcrafted features. However, the real-world images and videos consist of nonlinearity that must be used efficiently for gaining accuracies in the results. As in many other computer vision areas, deep learning-based methods have taken giant strides for obtaining state-of-the-art performance in crowd behaviour analysis. This paper presents a comprehensive survey of current convolution neural network (CNN)-based methods for crowd behaviour analysis. We have also surveyed popular software tools for CNN in the recent years. This survey presents detailed attributes of CNN with special emphasis on optimization methods that have been utilized in CNN-based methods. It also reviews fundamental and innovative methodologies, both conventional and latest methods of CNN, reported in the last few years. We introduce a taxonomy that summarizes important aspects of the CNN for approaching crowd behaviour analysis. Details of the proposed architectures, crowd analysis needs and their respective datasets are reviewed. In addition, we summarize and discuss the main works proposed so far with particular interest on CNNs on how they treat the temporal dimension of data, their highlighting features and opportunities and challenges for future research. To the best of our knowledge, this is a unique survey for crowd behaviour analysis using the CNN. We hope that this survey would become a reference in this ever-evolving field of research. [ABSTRACT FROM AUTHOR]

Visual tracking is an import topic in computer vision with many practical applications. Recently, deep learning methods have been introduced into the tracking community to improve tracking performance. How deep features affect tracking performance, however, has not been studied thoroughly. In this paper, we carry out an experimental study to deeply investigate the impact of convolutional features on tracking performance. We adopt the most influential and representative Convolutional Neural Network (CNN) models that are widely used in computer vision to equip our baseline tracking framework. Firstly, we carry out experiments on each CNN to reveal the relationship between tracking performance and different CNN layers. Secondly, we have a vertical comparison of tracking performance between different CNN models. In addition, we explore the effect of ensemble strategies of CNN features on tracking performance. Our work has got several valuable findings on the relationship between tracking performance and convolutional features. Based on our findings, we have derived a few useful guidelines for designing trackers with better performance. We have also developed a simple baseline tracker with the guidelines and it outperforms several state-of-the-art trackers very easily on challenging benchmark video sequences. [ABSTRACT FROM AUTHOR]