Search

People in crowd scenes always exhibit consistent behaviors and form collective motions. The analysis of collective motion has motivated a surge of interest in computer vision. Nevertheless, the effort is hampered by the complex nature of collective motions. Considering the fact that collective motions are formed by individuals, this paper proposes a new framework for both quantifying and detecting collective motion by investigating the spatio-temporal behavior of individuals. The main contributions of this work are threefold: 1) an intention-aware model is built to fully capture the intrinsic dynamics of individuals; 2) a structure-based collectiveness measurement is developed to accurately quantify the collective properties of crowds; 3) a multi-stage clustering strategy is formulated to detect both the local and global behavior consistency in crowd scenes. Experiments on real world data sets show that our method is able to handle crowds with various structures and time-varying dynamics. Especially, the proposed method shows nearly 10% improvement over the competitors in terms of NMI, Purity and RI. Its applicability is illustrated in the context of anomaly detection and semantic scene segmentation. [ABSTRACT FROM AUTHOR]

Abstract Weather Recognition plays an important role in our daily lives and many computer vision applications. However, recognizing the weather conditions from a single image remains challenging and has not been studied thoroughly. Generally, most previous works treat weather recognition as a single-label classification task, namely, determining whether an image belongs to a specific weather class or not. This treatment is not always appropriate, since more than one weather conditions may appear simultaneously in a single image. To address this problem, we make the first attempt to view weather recognition as a multi-label classification task, i.e., assigning an image more than one labels according to the displayed weather conditions. Specifically, a CNN–RNN based multi-label classification approach is proposed in this paper. The convolutional neural network (CNN) is extended with a channel-wise attention model to extract the most correlated visual features. The Recurrent Neural Network (RNN) further processes the features and excavates the dependencies among weather classes. Finally, the weather labels are predicted step by step. Besides, we construct two datasets for the weather recognition task and explore the relationships among different weather conditions. Experimental results demonstrate the superiority and effectiveness of the proposed approach. The new constructed datasets will be available at https://github.com/wzgwzg/Multi-Label-Weather-Recognition. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) is one of the most popular data representation methods in the field of computer vision and pattern recognition. High-dimension data are usually assumed to be sampled from the submanifold embedded in the original high-dimension space. To preserve the locality geometric structure of the data, k-nearest neighbor (k-NN) graph is often constructed to encode the near-neighbor layout structure. However, k-NN graph is based on Euclidean distance, which is sensitive to noise and outliers. In this article, we propose a refined-graph regularized nonnegative matrix factorization by employing a manifold regularized least-squares regression (MRLSR) method to compute the refined graph. In particular, each sample is represented by the whole dataset regularized with ℓ2-norm and Laplacian regularizer. Then a MRLSR graph is constructed based on the representative coefficients of each sample. Moreover, we present two optimization schemes to generate refined-graphs by employing a hard-thresholding technique. We further propose two refined-graph regularized nonnegative matrix factorization methods and use them to perform image clustering. Experimental results on several image datasets reveal that they outperform 11 representative methods. [ABSTRACT FROM AUTHOR]

Abstract: With the development of multimedia technology, image resizing has been raised as a question when the aspect ratio of an examined image should be displayed on a device with a different aspect ratio. Traditional nonuniform scaling for tackling this problem will lead to distortion. Therefore, content-aware consideration is mostly incorporated in the designing procedure. Such methods generally defines an energy function indicating the importance level of image content. The more important regions would be retained in the resizing procedure and distortion could be avoided consequently. The definition of the related energy function is thus the critical factor that directly influences the resizing results. In this work, we explore the definition of energy function from another aspect, matrix decomposition of Low-rank Representation. In our processing, a feature matrix that reflects the texture prior of object contour is firstly constructed. Then the matrix is decomposed into a low-rank one and sparse one. The energy function for resizing is then inferred from the sparse one. We illustrate the proposed method through seam carving framework and image shrinkage operation. Experiments on a dataset containing 1000 images demonstrate the effectiveness and robustness of the proposed method. [Copyright &y& Elsevier]

Abstract: Image segmentation is a fundamental problem in computer vision and image analysis. Specially, the segmentation of medical images can assist doctors in making decisions. Due to the lack of distinctive features to describe the boundary of an organ and match function with high performance for features, medical image segmentation is difficult to be achieved with high accuracy. In this paper, an one-class classifier is proposed as the match function to decide whether the pixel belongs to the boundary or not. The proposed method is comprised of two steps. At first, a feature vector space is built with the gradient feature and its statistical information in the training stage. In the test image, a feature vector of one candidate probably being located on the boundary is reconstructed by sparse coding with the feature vector space. After reconstruction, the candidate is classified belonging to boundary or non-boundary via the reconstruction based one-class classifier. Then, in order to maintain the consistency between the candidates which are neighbors to each other, the neighboring candidates are coded using group lasso with the same dictionary. Compared to the traditional methods, the proposed one has three advantages. Firstly, it solves the non-Gaussian distribution problem of the positive samples. Secondly, it avoids large variation among the negative samples. Thirdly, the relationship of the neighboring candidates is considered and used in classification, which is ignored in other methods. The proposed method is validated on 52 MR images of prostate and outperforms Mahalanobis distance, which is considered as one of the state-of-the-art match functions. The experimental results show that the segmentation accuracy can be significantly improved by the proposed method with one-class classification. [Copyright &y& Elsevier]

Abstract: The face almost always has an axis-symmetrical structure. However, as the face usually does not have an absolutely frontal pose when it is imaged, the majority of face images are not symmetrical images. These facts inspire us that the mirror image of the face image might be viewed as a representation of the face with a possible pose opposite to that of the original face image. In this paper we propose a scheme to produce the mirror image of the face and integrate the original face image and its mirror image for representation-based face recognition. This scheme is simple and computationally efficient. Almost all the representation-based classification methods can be improved by this scheme. The underlying rationales of the scheme are as follows: first, the use of the mirror image can somewhat overcome the misalignment problem of the face image in face recognition. Second, it is able to somewhat eliminate the side-effect of the variation of the pose and illumination of the original face image. The experiments show that the proposed scheme can greatly improve the accuracy of the representation-based classification methods. The proposed scheme might be also helpful for improving other face recognition methods. [Copyright &y& Elsevier]

Crowd localization is a new computer vision task, evolved from crowd counting. Different from the latter, it provides more precise location information for each instance, not just counting numbers for the whole crowd scene, which brings greater challenges, especially in extremely congested crowd scenes. In this paper, we focus on how to achieve precise instance localization in high-density crowd scenes, and to alleviate the problem that the feature extraction ability of the traditional model is reduced due to the target occlusion, the image blur, etc. To this end, we propose a Dilated Convolutional Swin Transformer (DCST) for congested crowd scenes. Specifically, a window-based vision transformer is introduced into the crowd localization task, which effectively improves the capacity of representation learning. Then, the well-designed dilated convolutional module is inserted into some different stages of the transformer to enhance the large-range contextual information. Extensive experiments evidence the effectiveness of the proposed methods and achieve the state-of-the-art performance on five popular datasets. Especially, the proposed model achieves F1-measure of 77.5% and MAE of 84.2 in terms of localization and counting performance, respectively. [ABSTRACT FROM AUTHOR]

Abstract: Multi-frame image super-resolution (SR) aims to utilize information from a set of low-resolution (LR) images to compose a high-resolution (HR) one. As it is desirable or essential in many real applications, recent years have witnessed the growing interest in the problem of multi-frame SR reconstruction. This set of algorithms commonly utilizes a linear observation model to construct the relationship between the recorded LR images to the unknown reconstructed HR image estimates. Recently, regularization-based schemes have been demonstrated to be effective because SR reconstruction is actually an ill-posed problem. Working within this promising framework, this paper first proposes two new regularization items, termed as locally adaptive bilateral total variation and consistency of gradients, to keep edges and flat regions, which are implicitly described in LR images, sharp and smooth, respectively. Thereafter, the combination of the proposed regularization items is superior to existing regularization items because it considers both edges and flat regions while existing ones consider only edges. Thorough experimental results show the effectiveness of the new algorithm for SR reconstruction. [Copyright &y& Elsevier]

Deep convolutional neural networks have achieved great success on many visual tasks (e.g., image classification). Non-linear activation plays a very important role in deep convolutional neural networks (CNN). It is found that the input distribution of non-linear activation is like Gaussian distribution and the most of the inputs are concentrated near zero. It makes the learned CNN likely sensitive to the small jitter of the non-linear activation input. Meanwhile, CNN is easily prone to overfitting with deep architecture. To solve the above problems, we make full use of the input distributions of non-linear activation and propose the randomly translational non-linear activation for deep CNN. In the training stage, non-linear activation function is randomly translated by an offset sampled from Gaussian distribution. In the test stage, the non-linear activation with zero offset is used. Based on our proposed method, the input distribution of non-linear activation is relatively scattered. As the result, the learned CNN is robust to the small jitter of the non-linear activation input. Our proposed method can be also seen as the regularization of non-linear activation to reduce overfitting. Compared to the original non-linear activation, our proposed method can improve classification accuracy without increasing computation cost. Experimental results on CIFAR-10/CIFAR-100, SVHN, and ImageNet demonstrate the effectiveness of the proposed method. For example, the reductions of error rates with VGG architecture on CIFAR-10/CIFAR-100 are 0.55% and 1.61%, respectively. Even when the noise is added to the input image, our proposed method still has much better classification accuracy on CIFAR-10/CIFAR-100. [ABSTRACT FROM AUTHOR]

With the coming of big data era, multi-view data represented by multiple features have been involved in many terms, such as machine learning, data mining and computer vision and so on. Due to the complex structure hidden in data, how to utilize the complementary and correlative information among multiple view features to improve classification performance is a challenging task. Moreover, the another challenging task is how to assign an appropriate weight for each classifier on the basis of its performance. To solve above problems, we proposed a supervised multi-view classification method based on Least Square Regression (LSR) and Ensemble Learning. To be specific, all samples for each view firstly can be classified by using Multi-class Support Vector Machine (MSVM); Then, to evaluate the classification results of different views for each sample, the optimal weight of each sample classification result is learned; Furthermore, considering the view difference with different classification quality, the view weights are assigned adaptively; Finally, we adopt the decision function values to determine the final classification results. Extensive experimental results show that the proposed method outperforms most state-of-the-art multi-view classification methods. [ABSTRACT FROM AUTHOR]

Visual tracking is a very critical issue in computer vision and video processing. For Discriminative Correlation Filter (DCF)-based tracking methods, it is very essential and meaningful to adaptively incorporate reliable target and surrounding information from video frames. However, most existing DCF-based trackers solely rely on pre-defined and fixed constraints such as a binary mask or quadratic function-based regularization to improve the discrimination. Unfortunately, such attempts fail to adjust the constraints according to the change of tracking circumstance in the video sequence, and thus lead to the lack of reliability of learned filters. To mitigate these problems, we present a novel DCF-based tracking method that introduces an adaptive target-and-surrounding soft mask (ATSM) into the learning formula. The adaptive soft mask that is represented by float numbers contains the detail information for both target region and its surrounding information: first, for the background area, it introduces meaningful background information and suppressing uninformative one; second, for the target area inside the bounding box, it helps to focus on the reliable area and repress the rapidly changing area; third, the target-and-surrounding soft mask is adaptively adjusted based on the variations of the target and its surrounding during the tracking process. By jointly modeling the filter and the adaptive soft mask, our ATSM tracker achieves an efficient integration of meaningful information of both foreground and background and performs favorably against state-of-the-art algorithms on seven well-known benchmarks. [ABSTRACT FROM AUTHOR]

Detecting moving objects from the video sequences has been treated as a challenging computer vision task, since the problems of dynamic background, multi-scale moving objects and various noise interference impact the corresponding feasibility and efficiency. In this paper, a novel spatio-temporal online matrix factorization (STOMF) method is proposed to detect multi-scale moving objects under dynamic background. To accommodate a wide range of the real noise distractions, we apply a specific mixture of exponential power (MoEP) distributions to the framework of low-rank matrix factorization (LRMF). For the optimization of solution algorithm, a temporal difference motion prior (TDMP) model is proposed, which estimates the motion matrix and calculates the weight matrix. Moreover, a partial spatial motion information (PSMI) post-processing method is further designed to implement multi-scale objects extraction in varieties of complex dynamic scenes, which utilizes partial background and motion information. The superiority of the STOMF method is validated by massive experiments on practical datasets, as compared with state-of-the-art moving objects detection approaches. [ABSTRACT FROM AUTHOR]

Abstract: In computer vision applications, Active Appearance Models (AAMs) is usually used to model the shape and the gray-level appearance of an object of interest using statistical methods, such as PCA. However, intensity values used in standard AAMs cannot provide enough information for image alignment. In this paper, we firstly propose to utilize Gabor filters to represent the image texture. The benefit of Gabor-based representation is that it can express local structures of an image. As a result, this representation can lead to more accurate matching when condition changes. Given the problem of the excessive storage and computational complexity of the Gabor, three different Gabor-based image representations are used in AAMs: (1) GaborD is the sum of Gabor filter responses over directions, (2) GaborS is the sum of Gabor filter responses over scales, and (3) GaborSD is the sum of Gabor filter responses over scales and directions. Through a large number of experiments, we show that the proposed Gabor representations lead to more accurate and robust matching between model and images. [Copyright &y& Elsevier]

• A novel nonlinear feature selective networks is designed. • Proposed method is applied for continuous values regression. • An optimizer joins back propagation and re-weighted is designed. The curse of dimensionality is a long-standing intractable issue in many machine learning and computer vision tasks. Feature selection, a data preprocessing technique, aims to select most discriminative feature subsets for improving performance of downstream machine learning tasks. However, most existing feature selection methods concentrate upon learning a linear relationship between data points and their labels, which causes that they are incapable of handling nonlinear complex data in real-world applications. In this paper, we first propose a novel end-to-end Nonlinear Feature Selective Networks (NFSN) that be able to select discriminative feature subsets while preserving their nonlinear structure by embedding a ℓ 2 , p -norm regularized hidden layer into designed continuous values regression networks. In addition, we propose an efficient optimization algorithm that joins back propagation algorithm and re-weighted optimization strategy to acquire derivative of all weights accurately. Experimental results on the nonlinear analog pulse signal and real-world datasets demonstrate the superiority of proposed method compared to some related methods on feature selection. Our source code available on: https://github.com/StevenWangNPU/NFSN. [ABSTRACT FROM AUTHOR]

Segmentation of the breast ultrasound (BUS) image is an important step for subsequent assessment and diagnosis of breast lesions. Recently, Deep-learning-based methods have achieved satisfactory performance in many computer vision tasks, especially in medical image segmentation. Nevertheless, those methods always require a large number of pixel-wise labeled data that is expensive in medical practices. In this study, we propose a new segmentation method by dense prediction and local fusion of superpixels for breast anatomy with scarce labeled data. First, the proposed method generates superpixels from the BUS image enhanced by histogram equalization, a bilateral filter, and a pyramid mean shift filter. Second, using a convolutional neural network (CNN) and distance metric learning-based classifier, the superpixels are projected onto the embedding space and then classified by calculating the distance between superpixels’ embeddings and the centers of categories. By using superpixels, we can generate a large number of training samples from each BUS image. Therefore, the problem of the scarcity of labeled data can be better solved. To avoid the misclassification of the superpixels, $K$ -nearest neighbor (KNN) is used to reclassify the superpixels within every local region based on the spatial relationships among them. Fivefold cross-validation was taken and the experimental results show that our method outperforms several often used deep-learning methods under the condition of the absence of a large number of labeled data (48 BUS images for training and 12 BUS images for testing). [ABSTRACT FROM AUTHOR]

The deformation and fracture of surrounding rock in deep underground engineering have the characteristics of time mutation and space continuity. In order to monitor and provide early warning of the dynamic deformation and fracture characteristics of deep surrounding rock, a real‐time digital displacement measuring instrument was developed in this paper. The proposed device represented a kind of surrounding rock deformation and fracture monitoring method, which had the continuous and real‐time advantages. Based upon the simulation experiments, the dynamic deformation and fracture of deep roadway surrounding rock were measured synchronously using the digital displacement measuring instrument testing system (DDMITS) and the V‐STARS measurement system. Comparative measurement results indicated that the DDMITS had a higher reliability. The deformation of roadway surrounding rock gradually increased with the increase of loading. The roof deformation and caving changing patterns in coal seam mining were studied using similar simulation experiments based on the DDMITS. The vertical displacement response of rock strata was more obvious than the horizontal displacement. There was a sharp increase in the vertical displacement during the collapse of the rock strata. Additionally, the vertical displacement velocity showed fluctuations. Meanwhile, the fluctuations in the vertical displacement acceleration increased significantly. The DDMITS could achieve full‐field measurement and the noncontact measurement of three‐dimensional deformation. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have achieved great success in several face-related tasks, such as face detection, alignment and recognition. As a fundamental problem in computer vision, face tracking plays a crucial role in various applications, such as video surveillance, human emotion detection and human-computer interaction. However, few CNN-based approaches are proposed for face (bounding box) tracking. In this article, we propose a face tracking method based on Siamese CNNs, which takes advantages of powerful representations of hierarchical CNN features learned from massive face images. The proposed method captures discriminative face information at both local and global levels. At the local level, representations for attribute patches (i.e., eyes, nose and mouth) are learned to distinguish a face from another one, which are robust to pose changes and occlusions. At the global level, representations for each whole face are learned, which take into account the spatial relationships among local patches and facial characters, such as skin color and nevus. In addition, we build a new large-scale challenging face tracking dataset to evaluate face tracking methods and to facilitate the research forward in this field. Extensive experiments on the collected dataset demonstrate the effectiveness of our method in comparison to several state-of-the-art visual tracking methods. [ABSTRACT FROM AUTHOR]

With the explosive growth of data, the multi-view data is widely used in many fields, such as data mining, machine learning, computer vision, and so on. Because such data always has a complex structure, i.e., many categories, many perspectives of description and high dimension, how to formulate an accurate and reliable framework for the multi-view classification is a very challenging task. In this paper, we propose a novel multi-view classification method by using multiple multi-class Support Vector Machines (SVMs) with a novel collaborative strategy. Here, each multi-class SVM embeds the scaling factor to renewedly adjust the weight allocation of all features, which is beneficial to highlight more important and discriminative features. Furthermore, we adopt the decision function values to integrate multiple multi-class learners and introduce the confidence score across multiple classes to determine the final classification result. In addition, through a series of the mathematical deduction, we bridge the proposed model with the solvable problem and solve it through an alternating iteration optimization method. We evaluate the proposed method on several image and face datasets, and the experimental results demonstrate that our proposed method performs better than other state-of-the-art learning algorithms. [ABSTRACT FROM AUTHOR]

This paper addresses two fundamental problems: 1) learning discriminative model parameters and 2) avoiding over-fitting, which often occurs in regression-based classification tasks. We formulate these two problems in terms of relaxing both the strict binary label matrix and graph regularization term into more flexible forms so that the margins between different classes are enlarged as much as possible and the problem of over-fitting is avoided to some extent. This task is accomplished by the proposed double relaxed regression (DRR) method. The convex problem of DRR is solved efficiently with an iterative procedure. Extensive experiments on synthetic and real world image data sets demonstrate the effectiveness of the proposed method in terms of both classification accuracy and running time. [ABSTRACT FROM AUTHOR]

• We proposed a novel Robust Matrix Completion method based on Rank-One matrix approximation (RMCRO), which divides the noisy and incomplete data matrix into two parts: a low-rank matrix and a sparse matrix, and it is robust to sparse noise. • We use a rank-one matrix pursuit method to approximate the low-rank part of the incomplete data matrix. Therefore, the proposed algorithm can recover the missing entries of the incomplete data matrix and estimate the rank simultaneously. • Extensive experimental results on both synthetic datasets and real world datasets demonstrate the effectiveness of the proposed method for matrix completion and rank estimation. Matrix completion aims at estimating the missing entries of a low-rank and incomplete data matrix. It frequently arises in many applications such as computer vision, pattern recognition, recommendation system, and data mining. Most of the existing methods face two problems. Firstly, the data matrix in real world is often disturbed by noise. Noise may change the date structure of the incomplete matrix, thereby degrade the performance of matrix completion algorithms. Secondly, some existing methods need to preset a reasonable rank as input, and the value of rank will affect the performance of the algorithms. Therefore, we proposed a robust rank-one matrix completion method with rank estimation in this paper. To mitigate the influence of noise, we divide the incomplete and noisy data matrix into two parts iteratively: low-rank and sparse parts. Besides, we use a weighted rank-one matrix pursuit algorithm to approximate the low-rank part of the data matrix, and the rank of the matrix can be estimated with the adaptive weight vector. The performance of the proposed method is demonstrated by experiments on both synthetic datasets and image datasets. The experimental results demonstrate the performance of the proposed method with incompleted matrices distrubed by sparse noise. [ABSTRACT FROM AUTHOR]

Inspired by recent development of artificial satellite, remote sensing images have attracted extensive attention. Recently, notable progress has been made in scene classification and target detection. However, it is still not clear how to describe the remote sensing image content with accurate and concise sentences. In this paper, we investigate to describe the remote sensing images with accurate and flexible sentences. First, some annotated instructions are presented to better describe the remote sensing images considering the special characteristics of remote sensing images. Second, in order to exhaustively exploit the contents of remote sensing images, a large-scale aerial image data set is constructed for remote sensing image caption. Finally, a comprehensive review is presented on the proposed data set to fully advance the task of remote sensing caption. Extensive experiments on the proposed data set demonstrate that the content of the remote sensing image can be completely described by generating language descriptions. The data set is available at https://github.com/201528014227051/RSICD_optimal. [ABSTRACT FROM AUTHOR]

Co-saliency detection is a newly emerging and rapidly growing research area in the computer vision community. As a novel branch of visual saliency, co-saliency detection refers to the discovery of common and salient foregrounds from two or more relevant images, and it can be widely used in many computer vision tasks. The existing co-saliency detection algorithms mainly consist of three components: extracting effective features to represent the image regions, exploring the informative cues or factors to characterize co-saliency, and designing effective computational frameworks to formulate co-saliency. Although numerous methods have been developed, the literature is still lacking a deep review and evaluation of co-saliency detection techniques. In this article, we aim at providing a comprehensive review of the fundamentals, challenges, and applications of co-saliency detection. Specifically, we provide an overview of some related computer vision works, review the history of co-saliency detection, summarize and categorize the major algorithms in this research area, discuss some open issues in this area, present the potential applications of co-saliency detection, and finally point out some unsolved challenges and promising future works. We expect this review to be beneficial to both fresh and senior researchers in this field and to give insights to researchers in other related areas regarding the utility of co-saliency detection algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we propose a unified co-salient object detection framework by introducing two novel insights: (1) looking deep to transfer higher-level representations by using the convolutional neural network with additional adaptive layers could better reflect the sematic properties of the co-salient objects; (2) looking wide to take advantage of the visually similar neighbors from other image groups could effectively suppress the influence of the common background regions. The wide and deep information are explored for the object proposal windows extracted in each image. The window-level co-saliency scores are calculated by integrating the intra-image contrast, the intra-group consistency, and the inter-group separability via a principled Bayesian formulation and are then converted to the superpixel-level co-saliency maps through a foreground region agreement strategy. Comprehensive experiments on two existing and one newly established datasets have demonstrated the consistent performance gain of the proposed approach. [ABSTRACT FROM AUTHOR]

Assessing aesthetic appeal of images is a highly subjective task which has attracted a lot of interests recently. It is an interdisciplinary subject related to art, psychology, and computer vision. In this paper, we systematically study prior researches of feature extraction in this area, and category them into four groups, low level, rule based, information theory, and visual attention. In each group, the effectiveness and limitations of existing features are examined. Based on the analysis, we propose a comprehensive feature set, which include 16 novel features and 70 well proved features. With this feature set, we build the system under machine learning scheme consisting of an SVM based classifier to estimate if an image is high aesthetic or low aesthetic . The experiments are conducted on public datasets show that our comprehensive feature set outperforms conventional models that concentrate mainly on certain types of features. The combination of our features produces a promising classification accuracy of 82.4% and a good performance comparable to aesthetic rating of human. Finally, we implemented the proposed evaluation system on mobile devices. It can provide real-time feedback to help users capture appealing photos. [ABSTRACT FROM AUTHOR]

Realistic action recognition has been one of the most challenging research topics in computer vision. The existing methods are commonly based on non-probabilistic classification, predicting category labels but not providing an estimation of uncertainty. In this paper, we propose a probabilistic framework using Gaussian processes (GPs), which can tackle regression problems with explicit uncertain models, for action recognition. A major challenge for GPs when applied to large-scale realistic data is that a large covariance matrix needs to be inverted during inference. Additionally, from the manifold perspective, the intrinsic structure of the data space is only constrained by a local neighborhood and data relationships with far-distance usually can be ignored. Thus, we design our GPs covariance matrix via the proposed ℓ 1 construction and a local approximation (LA) covariance weight updating method, which are demonstrated to be robust to data noise, automatically sparse and adaptive to the neighborhood. Extensive experiments on four realistic datasets, i.e., UCF YouTube, UCF Sports, Hollywood2 and HMDB51, show the competitive results of ℓ 1 -GPs compared with state-of-the-art methods on action recognition tasks. [ABSTRACT FROM AUTHOR]

Abstract: Flying bird detection (FBD) is critical in avoiding bird–aircraft collisions. Most existing approaches rely on motion detection to identify the flying bird, since it is a typical moving object. However, when there exist other moving objects, those methods often fail to distinguish flying birds from those objects due to the insufficiency of feature description. In this paper, we introduce a novel hierarchical feature model exploiting shape and shape dynamics to improve the ability of representing a flying bird, and then apply it to the FBD problem. As the shape of a flying bird is very distinctive in geometric structures and could provide discriminating spatial information, an improved shape context feature descriptor is proposed at the lower level to capture the spatial relations in bird shape. Then the shape descriptor is extended into the spatio-temporal domain and a shape dynamics description is built at the higher level, in which a 4-state Markovian model is adopted and is learned from training sequences. Moreover, to build a mapping from the lower level to the higher level of the hierarchy, a shape similarity index (SSI) based matching mechanism is designed. We apply these two-level features for detecting flying bird for improved safety of aircrafts flying at low-altitude. The experimental results show that the proposed method is effective and outperforms three other existing vision-based FBD approaches. [Copyright &y& Elsevier]

This paper comprehensively surveys the development of face hallucination (FH), including both face super-resolution and face sketch-photo synthesis techniques. Indeed, these two techniques share the same objective of inferring a target face image (e.g. high-resolution face image, face sketch and face photo) from a corresponding source input (e.g. low-resolution face image, face photo and face sketch). Considering the critical role of image interpretation in modern intelligent systems for authentication, surveillance, law enforcement, security control, and entertainment, FH has attracted growing attention in recent years. Existing FH methods can be grouped into four categories: Bayesian inference approaches, subspace learning approaches, a combination of Bayesian inference and subspace learning approaches, and sparse representation-based approaches. In spite of achieving a certain level of development, FH is limited in its success by complex application conditions such as variant illuminations, poses, or views. This paper provides a holistic understanding and deep insight into FH, and presents a comparative analysis of representative methods and promising future directions. [ABSTRACT FROM AUTHOR]

Abstract: The technique of visual saliency detection supports video surveillance systems by reducing redundant information and highlighting the critical, visually important regions. It follows that information about the image might be of great importance in depicting the visual saliency. However, the majority of existing methods extract contrast-like features without considering the contribution of information content. Based on the hypothesis that information divergence leads to visual saliency, a two-stage framework for saliency detection, namely information divergence model (IDM), is introduced in this paper. The term “information divergence” is used to express the non-uniform distribution of the visual information in an image. The first stage is constructed to extract sparse features by employing independent component analysis (ICA) and difference of Gaussians (DoG) filter. The second stage improves the Bayesian surprise model to compute information divergence across an image. A visual saliency map is finally obtained from the information divergence. Experiments are conducted on nature image databases, psychological patterns and video surveillance sequences. The results show the effectiveness of the proposed method by comparing it with 13 state-of-the-art visual saliency detection methods. [Copyright &y& Elsevier]

Recovering a large matrix from a small subset of its entries is a challenging problem arising in many real applications, such as image inpainting and recommender systems. Many existing approaches formulate this problem as a general low-rank matrix approximation problem. Since the rank operator is nonconvex and discontinuous, most of the recent theoretical studies use the nuclear norm as a convex relaxation. One major limitation of the existing approaches based on nuclear norm minimization is that all the singular values are simultaneously minimized, and thus the rank may not be well approximated in practice. In this paper, we propose to achieve a better approximation to the rank of matrix by truncated nuclear norm, which is given by the nuclear norm subtracted by the sum of the largest few singular values. In addition, we develop a novel matrix completion algorithm by minimizing the Truncated Nuclear Norm. We further develop three efficient iterative procedures, TNNR-ADMM, TNNR-APGL, and TNNR-ADMMAP, to solve the optimization problem. TNNR-ADMM utilizes the alternating direction method of multipliers (ADMM), while TNNR-AGPL applies the accelerated proximal gradient line search method (APGL) for the final optimization. For TNNR-ADMMAP, we make use of an adaptive penalty according to a novel update rule for ADMM to achieve a faster convergence rate. Our empirical study shows encouraging results of the proposed algorithms in comparison to the state-of-the-art matrix completion algorithms on both synthetic and real visual datasets. [ABSTRACT FROM AUTHOR]

Abstract: Visual saliency detection has been applied in many tasks in the fields of pattern recognition and computer vision, such as image segmentation, object recognition, and image retargeting. However, the accurate detection of saliency remains a challenge. The reasons behind this are that: (1) well-defined mechanism for saliency definition is rarely established; and (2) supporting information for detecting saliency is limited in general. In this paper, a multi-spectrum based saliency detection algorithm is proposed. Instead of only using the conventional RGB information as what existing algorithms do, this work incorporates near-infrared clues into the detection framework. Features of color and texture from both types of image modes are explored simultaneously. When calculating the color contrast, an effective color component analysis method is employed to produce more precise results. With respect to the texture analysis, texton representation is adopted for fast processing. Experiments are done to compare the proposed algorithm with other 11 state-of-the-art algorithms and the results indicate that our algorithm outperforms the others. [Copyright &y& Elsevier]

Recently, much attention has been drawn to the problem of matrix completion, which arises in a number of fields, including computer vision, pattern recognition, sensor network, and recommendation systems. This paper proposes a novel algorithm, named robust alternative minimization (RAM), which is based on the constraint of low rank to complete an unknown matrix. The proposed RAM algorithm can effectively reduce the relative reconstruction error of the recovered matrix. It is numerically easier to minimize the objective function and more stable for large-scale matrix completion compared with other existing methods. It is robust and efficient for low-rank matrix completion, and the convergence of the RAM algorithm is also established. Numerical results showed that both the recovery accuracy and running time of the RAM algorithm are competitive with other reported methods. Moreover, the applications of the RAM algorithm to low-rank image recovery demonstrated that it achieves satisfactory performance. [ABSTRACT FROM AUTHOR]

The biologically inspired model (BIM) proposed by Serre presents a promising solution to object categorization. It emulates the process of object recognition in primates' visual cortex by constructing a set of scale- and position-tolerant features whose properties are similar to those of the cells along the ventral stream of visual cortex. However, BIM has potential to be further improved in two aspects: mismatch by dense input and randomly feature selection due to the feedforward framework. To solve or alleviate these limitations, we develop an enhanced BIM (EBIM) in terms of the following two aspects: 1) removing uninformative inputs by imposing sparsity constraints, 2) apply a feedback loop to middle level feature selection. Each aspect is motivated by relevant psychophysical research findings. To show the effectiveness of the EBIM, we apply it to object categorization and conduct empirical studies on four computer vision data sets. Experimental results demonstrate that the EBIM outperforms the BIM and is comparable to state-of-the-art approaches in terms of accuracy. Moreover, the new system is about 20 times faster than the BIM. [ABSTRACT FROM AUTHOR]

Eye detection plays an important role in many practical applications. This paper presents a novel two-step scheme for eye detection. The first step models an eye by a newly defined visual-context pattern (VCP), and the second step applies semisupervised boosting for precise detection. VCP describes both the space and appearance relations between an eye region (region of eye) and a reference region (region of reference). The context feature of a VCP is extracted by using the integral image. Aiming to reduce the human labeling efforts, we apply semisupervised boosting, which integrates the context feature and the Haar-like features for precise eye detection. Experimental results on several standard face data sets demonstrate that the proposed approach is effective, robust, and efficient. We finally show that this approach is ready for practical applications. [ABSTRACT FROM AUTHOR]

Abstract: With the popularity of cameras and rapid development of computer vision technology, vision-based HCI is attracting extensive interests. In this paper, we present a system for controlling avatars by natural body movement via a single web-camera. A pose database and a set of color markers are utilized to make ill-posed vision problem tractable for real game applications. Based on the proposed algorithms for indexing pose samples and estimating human pose, we build a prototype system that is responsive, easy to manipulate and runs automatically in real time. User study shows that the system is user-friendly and provides immersive experiences. [Copyright &y& Elsevier]

• A GA training pipeline to select key input channels of DenseNet automatically. • A 'Weight inheritance' strategy to improve efficiency whilst maintaining accuracy. • To reduce up to 30% parameters yet achieving a similar accuracy as the DenseNet. • Validated robustness/scalability on several image recognition/classification datasets. Convolutional neural networks (CNNs) have been successfully applied in many computer vision applications [1] , especially in image classification tasks, where most of the structures have been designed manually. With the aid of skip connection and dense connection, the depths of the models are becoming "deeper" and the filters of layers are getting "wider" in order to tackle the challenge of large-scale datasets. However, large-scale models in convolutional layers become inefficient due to the redundant channels from input feature maps. In this paper, we aim to automatically optimize the topology of the DenseNet, in which unnecessary convolutional kernels are reduced. To achieve this, we present a training pipeline that generates the network structure using a genetic algorithm. We first propose two encoding methods that can represent the structure of the model using a fixed-length binary string. A three-step based evolutionary process consisting of selection, crossover, and mutation is proposed to optimize the structure. We also present a pretrained weight inheritance method which can largely reduce the total time consumption of the genetic process. Experimental results have demonstrated that our proposed model can achieve comparable accuracy to the state-of-the-art models, across a wide range of image recognition and classification datasets, whilst significantly reducing the number of parameters. [ABSTRACT FROM AUTHOR]

• This paper proposes a novel stacked Squeeze-and-Excitation Recurrent Residual Network (SER2-Net) for visual-semantic matching. • This paper develops an effective and efficient cross-modal representation learning module, which is capable of generating semantically complementary multi-level features for both modalities. • This paper presents a novel objective function for aligning cross-modal data, which is able to capture the interdependency among multiple semantic levels to alleviate the distribution inconsistency between visual and textual modality. • Extensive experiments on two benchmark datasets demonstrate the superiority of our proposed model compared to the state-of-the-art approaches. In recent years, visual-textual matching has been widely studied in the intersection of computer vision and natural language processing communities. A feasible scheme for learning discriminative representations is leveraging hierarchical features to align both modalities at multiple semantic levels. However, most existing approaches rely on pre-trained object detectors or semantic parsers to generate multi-level representations, whose performance is overly dependent on the extra supervision and thereby leads to its vulnerability. In this paper, we introduce a Stacked Squeeze-and-Excitation Recurrent Residual Network (SER2-Net) for visual-textual matching. Firstly, an efficient multi-level representation module is presented to produce a series of semantically discriminative features without the aid of extra supervision, which is built by stacking the squeeze-and-excitation recurrent residual (SER2) learning components. Specifically, SER2 incorporates the residual learning and inverse recurrent connection into the squeeze-and-excitation learning block, which allows for utilizing complementary current information and residual information to improve the modality-specific representation ability. Besides, to capture the implicit correlations contained among multi-level features, we propose a novel objective namely Cross-modal Semantic Discrepancy (CMSD) loss, which is characterized by exploiting the interdependency among different semantic levels to narrow the cross-modal distribution discrepancy. Extensive experiments on two benchmark datasets validate the superiority of our model, which compares favorably with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The article discusses various reports published within the issue including the two schemes for Bidirectional Texture Function (BTF) data and a statistical method for choosing image patches for object detection and classification.