Search

Many representation methods were built for gray image textures. However, they are not effective for color textures in general. To alleviate this problem, in this paper we propose a novel Compact Interchannel Sampling Difference Descriptor (CISDD) for color texture classification. In particular, considering sampling-based method can capture more directional information, we first use a heavy-tailed distribution, t-distribution to generate sample points in the image patch to calculate the micro-block difference. Then we model the interchannel relationship of color texture image by using dense micro-block differences. Furthermore, we utilize principal component analysis (PCA) to reduce the dimensions of the features encoded by the Fisher vector, and construct a Compact Interchannel Sampling Difference Descriptor (CISDD) for representing color texture image. Finally, experimental results on five published standard texture datasets (KTH-TIPS, VisTex, CUReT, USPTex and Colored Brodatz) reveal that CISDD is effective and outperforms thirteen representative color texture classification methods. [ABSTRACT FROM AUTHOR]

Crowd counting from a single image is a challenging task due to high appearance similarity, perspective changes, and severe congestion. Many methods only focus on the local appearance features and they cannot handle the aforementioned challenges. In order to tackle them, we propose a perspective crowd counting network (PCC Net), which consists of three parts: 1) density map estimation (DME) focuses on learning very local features of density map estimation; 2) random high-level density classification (R-HDC) extracts global features to predict the coarse density labels of random patches in images; and 3) fore-/background segmentation (FBS) encodes mid-level features to segments the foreground and background. Besides, the Down, Up, Left, and Right (DULR) module is embedded in PCC Net to encode the perspective changes on four directions (DULR). The proposed PCC Net is verified on five mainstream datasets, which achieves the state-of-the-art performance on the one and attains the competitive results on the other four datasets. The source code is available at https://github.com/gjy3035/PCC-Net. [ABSTRACT FROM AUTHOR]

Existence of haze significantly degrades visual quality and hence negatively affects the performance of visual surveillance, video analysis, and human–machine interaction. To remove haze from a visual signal, in this paper, we propose a generative adversarial network for visual haze removal called HRGAN. HRGAN consists of a generator network and a discriminator network. A unified network jointly estimating transmission maps, atmospheric light, and haze-free images (called UNTA) is proposed as the generator network of HRGAN. Instead of being optimized by minimizing the pixel-wise loss, HRGAN is optimized by minimizing a novel loss function consisting of pixel-wise loss, perceptual loss, and adversarial loss produced by a discriminator network. Classical model-based image dehazing algorithms consist of three separate stages: 1) estimating transmission map; 2) estimating atmospheric light; and 3) restoring haze-free image by using an atmospheric scattering model to process the transmission map and atmospheric light. Such a separate scheme is not guaranteed to achieve optimal results. On the contrary, UNTA performs transmission map estimation and atmospheric light estimation simultaneously to obtain joint optimal solutions. The experimental results on both synthetic and real-world image databases demonstrate that HRGAN outperforms the state-of-the-art algorithms in terms of both effectiveness and efficiency. [ABSTRACT FROM AUTHOR]

Preserving projection-based methods are good for finding the manifold structure embedded in data. As they use the Euclidean distance as a metric, which is sensitive to noise and outliers in data, nuclear norm-based 2D locality preserving projection (NN-2DLPP) is thus proposed to improve the robustness of 2DLPP. However, NN-2DLPP does not consider the discriminant ability of data. In order to improve the discriminant ability of preserving projection methods, in this paper, we use preserving projection learning with structurally incoherence of data and propose structurally incoherent low-rank 2DLPP (SILR-2DLPP) for image classification. This approach provides a discriminative representation of preserving projection learning by recovering the distinct different classes of the data. SILR-2DLPP searches the optimal subspace and low-rank representation simultaneously. We further extend SILR-2DLPP to a kernel case and propose kernel SILR-2DLPP (KSILR-2DLPP) to obtain a nonlinear representation. The theoretical analysis including the convergence and computational complexity of SILR-2DLPP are presented. To verify the performance of SILR-2DLPP and KSILR-2DLPP, six well-known image databases were used in the experiments. The experimental results show that the proposed methods are superior to the previous preserving projection methods for image classification. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF), known as a famous matrix factorization technique, has been widely used in pattern recognition and computer vision. NMF represents the input data matrix as a product of two nonnegative factors. As NMF is based on the Euclidean distance, which is sensitive to noise or errors in the data, some robust NMF methods are proposed. Mainly focusing on parts-based representation, these robust NMF methods often neglect global representation of data. In fact, the global geometry information of data is more robust than the local information about the noisy data in terms of image classification. In order to effectively improve the robustness of NMF and learn part-based and global representation of the data, a novel method low-rank nonnegative factorization (LRNF) is proposed in this paper. First, we assume that the data are grossly corrupted, and the $L_{1} $ norm is used as a sparse constraint on the assumed noise matrix. Then, LRNF learns a low-rank matrix with the global representation ability. Finally, we make a nonnegative factorization of the learned low-rank matrix. We can obtain a base matrix, which preserves locality and globality properties of the data in the meantime. Extensive experiments have been conducted on nine real-world image databases to verify the performance of the proposed LRNF method by comparing with the state-of-the-art algorithms on robust dimensionality reduction. [ABSTRACT FROM AUTHOR]

Moving object detection is a key to intelligent video analysis. On the one hand, what moves are not only interesting objects but also noise and cluttered background. On the other hand, moving objects without rich texture are prone to not be detected. Therefore, there are undesirable false alarms and missed alarms in the results of many algorithms of moving object detection. To reduce the false alarms and missed alarms, in this paper we propose to incorporate a saliency map into an incremental subspace analysis framework in which the saliency map makes the estimated background have less of a chance than the foreground (i.e., moving objects) to contain salient objects. The proposed objective function systematically takes into account the properties of sparsity, low rank, connectivity, and saliency. An alternative minimization algorithm is proposed to seek the optimal solutions. The experimental results on both the Perception Test Images Sequences data set and Wallflower data set demonstrate that the proposed method is effective in reducing false alarms and missed alarms. [ABSTRACT FROM PUBLISHER]

Visual tracking is a very critical issue in computer vision and video processing. For Discriminative Correlation Filter (DCF)-based tracking methods, it is very essential and meaningful to adaptively incorporate reliable target and surrounding information from video frames. However, most existing DCF-based trackers solely rely on pre-defined and fixed constraints such as a binary mask or quadratic function-based regularization to improve the discrimination. Unfortunately, such attempts fail to adjust the constraints according to the change of tracking circumstance in the video sequence, and thus lead to the lack of reliability of learned filters. To mitigate these problems, we present a novel DCF-based tracking method that introduces an adaptive target-and-surrounding soft mask (ATSM) into the learning formula. The adaptive soft mask that is represented by float numbers contains the detail information for both target region and its surrounding information: first, for the background area, it introduces meaningful background information and suppressing uninformative one; second, for the target area inside the bounding box, it helps to focus on the reliable area and repress the rapidly changing area; third, the target-and-surrounding soft mask is adaptively adjusted based on the variations of the target and its surrounding during the tracking process. By jointly modeling the filter and the adaptive soft mask, our ATSM tracker achieves an efficient integration of meaningful information of both foreground and background and performs favorably against state-of-the-art algorithms on seven well-known benchmarks. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel energy optimization framework to accurately estimate surface normal and reflectance of an object from an input image sequence. Input images are captured from a fixed viewpoint under varying lighting conditions. In the proposed approach we combine photometric stereo and Retinex constraints into our energy function. To formulate inter-image constraints, shading information is added to the Lambertian model to account for shadows. For intra-image constraints, we moderate the strength of shading smoothness according to shadow mask and normal variations. By minimizing this energy function we are able to recover accurate surface normals and reflectance. Experimental results show that our approach yields more realistic normal map and accurate albedo map than the state-of-the-art uncalibrated photometric stereo algorithms. As for intrinsic image decomposition, results on the real and synthetic scenes show that the proposed approach outperforms previous ones. [ABSTRACT FROM PUBLISHER]

Multimodal emotion recognition in conversational videos (ERC) develops rapidly in recent years. To fully extract the relative context from video clips, most studies build their models on the entire dialogues which make them lack of real-time ERC ability. Different from related researches, a novel multimodal emotion recognition model for conversational videos based on reinforcement learning and domain knowledge (ERLDK) is proposed in this paper. In ERLDK, the reinforcement learning algorithm is introduced to conduct real-time ERC with the occurrence of conversations. The collection of history utterances is composed as an emotion-pair which represents the multimodal context of the following utterance to be recognized. Dueling deep-Q-network (DDQN) based on gated recurrent unit (GRU) layers is designed to learn the correct action from the alternative emotion categories. Domain knowledge is extracted from public dataset based on the former information of emotion-pairs. The extracted domain knowledge is used to revise the results from the RL module and is transformed into other dataset to examine the rationality. The experimental results on datasets show that ERLDK achieves the state-of-the-art results on weighted average and most of the specific emotion categories. [ABSTRACT FROM AUTHOR]

Detecting moving objects from the video sequences has been treated as a challenging computer vision task, since the problems of dynamic background, multi-scale moving objects and various noise interference impact the corresponding feasibility and efficiency. In this paper, a novel spatio-temporal online matrix factorization (STOMF) method is proposed to detect multi-scale moving objects under dynamic background. To accommodate a wide range of the real noise distractions, we apply a specific mixture of exponential power (MoEP) distributions to the framework of low-rank matrix factorization (LRMF). For the optimization of solution algorithm, a temporal difference motion prior (TDMP) model is proposed, which estimates the motion matrix and calculates the weight matrix. Moreover, a partial spatial motion information (PSMI) post-processing method is further designed to implement multi-scale objects extraction in varieties of complex dynamic scenes, which utilizes partial background and motion information. The superiority of the STOMF method is validated by massive experiments on practical datasets, as compared with state-of-the-art moving objects detection approaches. [ABSTRACT FROM AUTHOR]

We present a novel interactive image segmentation approach with user scribbles using constrained Laplacian graph optimization. A novel energy framework is developed by adding the smoothing item in the cost function of Laplacian graph energy. To the best of our knowledge, our approach is the first to incorporate the normalized cuts and graph cuts algorithms into a unified energy optimization framework. The proposed approach is further accelerated by running the proposed optimization method on a band region when we segment the large images. Our acceleration strategy enables our approach to efficiently segment the large images, which yields about a 20–80 times speedup. The proposed approach is evaluated on both the publicly available data sets and our own data set with large images. The benefits of the proposed unified framework are also demonstrated both qualitatively and quantitatively. The experimental results show that our segmentation method achieves better performance of both boundary recall and error rate than the existing state-of-the-art approaches. [ABSTRACT FROM PUBLISHER]

As digital video databases grow, so grows the problem of effectively navigating through them. In this paper we propose a novel content-based video retrieval approach to searching such video databases, specifically those involving human actions, incorporating spatio-temporal localization. We outline a novel, highly efficient localization model that first performs temporal localization based on histograms of evenly spaced time-slices, then spatial localization based on histograms of a 2-D spatial grid. We further argue that our retrieval model, based on the aforementioned localization, followed by relevance ranking, results in a highly discriminative system, while remaining an order of magnitude faster than the current state-of-the-art method. We also show how relevance feedback can be applied to our localization and ranking algorithms. As a result, the presented system is more directly applicable to real-world problems than any prior content-based video retrieval system. [ABSTRACT FROM PUBLISHER]

Single-stage detectors have gained great attention due to their high detection accuracy and real-time speed. To detect multi-scale objects, single-stage detectors make scale-aware predictions based on multiple pyramid layers. However, the insufficient context exploration in shallow pyramid layers leads to the detection accuracy of small objects being far from satisfactory. To tackle this problem, we propose a scheme to selectively extract multi-scale context with attention-adaptive weights. Specifically, we propose an efficient selective context network for accurate object detection. It incorporates an enhanced context module and a triple attention module. The enhanced context module consists of multi-branches to extract original-scale, small-scale, and large-scale contextual information. To make full use of this context and filter out noisy information, the triple attention module, which contains global-level, channel-level, and spatial-level attentions, is introduced to carry out selective context fusion. The two modules are easy to implement and can efficiently boost the accuracy of object detection. The performance of our method is validated on two benchmarks: PASCAL VOC and MS COCO. For a $512\times 512$ input, our detector with VGG16 achieves competitive results (80.9 on the Pascal VOC 2012 test set in the case of single-scale inference without MS COCO pre-training). On the MS COCO test-dev set, our detector with ResNet101 outperforms RetinaNet500 by 2.5% AP in terms of overall performance and its speed is 48 milliseconds on a Titan XP GPU. As a result, ESCNet achieves a better trade-off between accuracy and speed. [ABSTRACT FROM AUTHOR]

To better solve scale variance problem, deep multi-scale methods usually detect objects of different scales by different in-network layers. However, the semantic levels of features from different layers are usually inconsistent. In this paper, we propose a multi-branch and high-level semantic network by gradually splitting a base network into multiple different branches. As a result, the different branches have same depth and the output features of different branches have similarly high-level semantics. Due to the difference of receptive fields, the different branches are suitable to detect objects of different scales. Meanwhile, the multi-branch network does not introduce additional parameters by sharing the convolutional weights of different branches. To further improve detection performance, skip-layer connections are used to add context to the branch of relatively small receptive field, and dilated convolution is incorporated to enlarge the resolutions of output feature maps. When they are embedded into Faster RCNN architecture, the weighted scores of proposal generation network and proposal classification network are further proposed. Experiments on three pedestrian datasets (i.e., the KITTI dataset, the Caltech dataset, and the Citypersons dataset), one face dataset (i.e., the WIDER FACE dataset), and two general object datasets (i.e., the COCO benchmark and the PASCAL VOC dataset) demonstrate the effectiveness and generality of proposed method. On these datasets, our method achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

The human action is an important information source for human social interaction, and it simultaneously plays a crucial role in human–robot interaction (HRI). For a natural and fluent interaction, robots are required to understand human actions and have the capacity to predict action intentions and to imitate human actions for an appropriate response. Currently, existing survey papers for the action recognition mainly summarize algorithms that perform action recognition in experimental scenarios, and survey papers of the HRI mainly introduced various interaction interfaces in the HRI. Different from these surveys, we focus on the human action analysis on robot platforms for the HRI application, including the body motion and gestures. We review the existing HRI related references involving the action recognition, prediction, and the robot imitation of the human action. Moreover, we give a summary of robot platforms and action datasets that are frequently used in the study of HRI. Finally, we give an analysis on the development trend and future research directions of action analysis for the HRI applications. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of supervised video summarization by formulating it as a sequence-to-sequence learning problem, where the input is a sequence of original video frames, and the output is a keyshot sequence. Our key idea is to learn a deep summarization network with attention mechanism to mimic the way of selecting the keyshots of human. To this end, we propose a novel video summarization framework named attentive encoder–decoder networks for video summarization (AVS), in which the encoder uses a bidirectional long short-term memory (BiLSTM) to encode the contextual information among the input video frames. As for the decoder, two attention-based LSTM networks are explored by using additive and multiplicative objective functions, respectively. Extensive experiments are conducted on two video summarization benchmark datasets, i.e., SumMe and TVSum. The results demonstrate the superiority of the proposed AVS-based approaches against the state-of-the-art approaches, with remarkable improvements on both datasets. [ABSTRACT FROM AUTHOR]

This paper addresses two fundamental problems: 1) learning discriminative model parameters and 2) avoiding over-fitting, which often occurs in regression-based classification tasks. We formulate these two problems in terms of relaxing both the strict binary label matrix and graph regularization term into more flexible forms so that the margins between different classes are enlarged as much as possible and the problem of over-fitting is avoided to some extent. This task is accomplished by the proposed double relaxed regression (DRR) method. The convex problem of DRR is solved efficiently with an iterative procedure. Extensive experiments on synthetic and real world image data sets demonstrate the effectiveness of the proposed method in terms of both classification accuracy and running time. [ABSTRACT FROM AUTHOR]

In the past few years, scene contextual information has been increasingly used for action understanding with promising results. However, unsupervised video action clustering using context has been less explored, and existing clustering methods cannot achieve satisfactory performances. In this paper, we propose a novel unsupervised video action clustering method by using the motion-scene interaction constraint (MSIC). The proposed method takes the unique static scene and dynamic motion characteristics of video action into account, and develops a contextual interaction constraint model under a self-representation subspace clustering framework. First, the complementarity of multi-view subspace representation in each context is explored by single-view and multi-view constraints. Afterward, the context-constrained affinity matrix is calculated and the MSIC is introduced to mutually regularize the disagreement of subspace representation in scene and motion. Finally, by jointly constraining the complementarity of multi-views and the consistency of multi-contexts, an overall objective function is constructed to guarantee the video action clustering result. The experiments on four video benchmark datasets (Weizmann, KTH, UCFsports, and Olympic) demonstrate that the proposed method outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

2-D linear discriminant analysis (2DLDA) has been widely used in pattern recognition and image classification. 2DLDA selects discriminative features from the up and left corner of images. However, 2DLDA uses the Frobenius norm (F-norm), which is sensitive to noise or outliers in data, as a metric. In this paper, we propose a novel framework, called horizontal and vertical nuclear norm-based 2DLDA (HVNN-2DLDA) for image representation. In the proposed framework, HVNN-2DLDA methods (i.e., HNN-2DLDA and VNN-2DLDA) are proposed, and both use the nuclear norm as a criterion. The nuclear norm can provide more structure and global information for the reconstruction of noisy images. HNN-2DLDA and VNN-2DLDA represent images in the row and column directions, respectively. In addition, by combining the row and column directions, we propose a bilateral nuclear norm-based 2DLDA method called BNN-2DLDA. The advantage of BNN-2DLDA over HNN-2DLDA and VNN-2DLDA is that an image sample can be represented by both the row and the column directions instead of only the row or column direction. HVNN-2DLDA learns a set of local optimal projection vectors by maximizing the ratio of the nuclear norm of the between-class scatter matrix and the nuclear norm of the within-class scatter matrix. To verify the robustness and recognition performance in image classification of HVNN-2DLDA, six public image databases are used for experiments. The experimental results demonstrate the effectiveness and the feasibility of the proposed framework. [ABSTRACT FROM AUTHOR]

Dynamic scene classification has been extensively studied in computer vision due to its widespread applications. The key to dynamic scene classification lies in jointly characterizing spatial appearance and temporal dynamics to achieve informative representation, which remains an outstanding task in the literature. In this paper, we propose a unified framework to extract spatial and temporal features for dynamic scene representation. More specifically, we deploy two variants of deep convolutional neural networks to encode spatial appearance and short-term dynamics into short-term deep features (STDF). Based on STDF, we propose using the autoregressive moving average model to extract long-term frequency features (LTFF). By combining STDF and LTFF, we establish the long–short-term feature (LSTF) representations of dynamic scenes. The LSTF characterizes both spatial and temporal patterns of dynamic scenes for comprehensive and information representation that enables more accurate classification. Extensive experiments on three-dynamic scene classification benchmarks have shown that the proposed LSTF achieves high performance and substantially surpasses the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In the field of 3-D technology, it is interesting as well as meaningful issue to control object depth in 3-D space. Recently, some depth control methods for stereoscopic images have been proposed, which usually employ depth map or directly process color images to implement depth control. There are two main disadvantages for these methods. First, the results of these methods usually suffer from object deformation and holes. Second, these methods are prone to cause undesired object size changing in 3-D space. To address these issues, we propose a shape-preserving object depth control method for stereoscopic images. First, a novel depth mapping model is presented for calculating the ideal coordinates of the key points in depth control, so that the shape of the object can be well preserved. Afterward, the image content-based constraints are used to further preserve the structure of the object and its background. Finally, the warping technology is introduced to deal with images optimally as well as to avoid holes. Experimental results show that the proposed method can control object depth and preserve the shape of the object effectively without sensible background distortion. [ABSTRACT FROM AUTHOR]

Traffic videos are often recorded by vehicle-mounted cameras. Compared with videos recorded by handheld cameras, traffic videos suffer from more challenges, such as higher frequency and more violent jitters, dynamic scenes, large moving objects, and parallax, which can result in significant visual quality degradation. To address these challenges for traffic videos, we propose a special stabilization method. The key aspect of our method is a feedback strategy that divides the extracted feature trajectories into background trajectories and foreground trajectories by feeding back the previous trajectory classification results. The method can perform robustly, even in the case of large moving objects and parallax. Furthermore, our method maintains the number of available background trajectories within a reasonable range via two refinement strategies. One strategy attempts to reliably recover background trajectories from misjudged foreground trajectories when there are an insufficient number of background trajectories. The other strategy can adaptively adjust the number of feature points in each frame to efficiently avoid too many or too few background trajectories. With the obtained background trajectories, a homography matrix between each frame and its stabilized view is computed and implemented to warp the frame image to produce smooth videos. Experimental results confirm that our method is both effective in stabilizing traffic videos and quite robust against large moving objects and parallax. [ABSTRACT FROM PUBLISHER]

Nonnegative matrix factorization (NMF), which aims at obtaining the nonnegative low-dimensional representation of data, has received wide attention. To obtain more effective nonnegative discriminant bases from the original NMF, in this paper, a novel method called nonnegative discriminant matrix factorization (NDMF) is proposed for image classification. NDMF integrates the nonnegative constraint, orthogonality, and discriminant information in the objective function. NDMF considers the incoherent information of both factors in standard NMF and is proposed to enhance the discriminant ability of the learned base matrix. NDMF projects the low-dimensional representation of the subspace of the base matrix to regularize the NMF for discriminant subspace learning. Based on the Euclidean distance metric and the generalized Kullback–Leibler (KL) divergence, two kinds of iterative algorithms are presented to solve the optimization problem. The between- and within-class scatter matrices are divided into positive and negative parts for the update rules and the proofs of the convergence are also presented. Extensive experimental results demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art discriminant NMF algorithms. [ABSTRACT FROM PUBLISHER]

With the rapid development of the mobile service and online social networking service, a large number of mobile images are generated and shared on the social networks every day. The visual content of these images contains rich knowledge for many uses, such as social categorization and recommendation. Mobile image labeling has, therefore, been proposed to understand the visual content and received intensive attention in recent years. In this paper, we present a novel mobile image labeling scheme on the cloud, in which mobile images are first and efficiently transmitted to the cloud by Hamming compressed sensing, such that the heavy computation for image understanding is transferred to the cloud for quick response to the queries of the users. On the cloud, we design a sparse correntropy framework for robustly learning the semantic content of mobile images, based on which the relevant tags are assigned to the query images. The proposed framework (called maximum correntropy-based mobile image labeling) is very insensitive to the noise and the outliers, and is optimized by a half-quadratic optimization technique. We theoretically show that our image labeling approach is more robust than the squared loss, absolute loss, Cauchy loss, and many other robust loss function-based sparse coding methods. To further understand the proposed algorithm, we also derive its robustness and generalization error bounds. Finally, we conduct experiments on the PASCAL VOC’07 data set and empirically demonstrate the effectiveness of the proposed robust sparse coding method for mobile image labeling. [ABSTRACT FROM PUBLISHER]

The increasing popularity of online social networks (OSNs) has been transforming the dissemination pattern of social video contents. We can utilize the social information propagation pattern to improve the efficiency of social video distribution. In this paper, motivated by the social community classification, we present a social video replication and user request dispatching mechanism in the cloud content delivery network architecture to reduce the system operational cost, while guaranteeing the averaged service latency. Specifically, we first present a community classification method that clusters social users with social relationships, close geolocations, and similar video watching interests into various communities. Then, we conduct a large-scale measurement on a real OSN system to study the diversities of social video propagation and the effectiveness of our communities on smoothing the diversity. Finally, we propose the community-based video replication and request dispatching strategy and formulate it as a constrained optimization problem. Based on a stochastic optimization framework, we derive an online solution and rigorously prove the optimality. We evaluate our algorithm on a real trace under realistic settings and demonstrate that our algorithm can reduce the monetary cost by 30% against traditional approaches with the same service latency. [ABSTRACT FROM AUTHOR]

Background information is crucial for many video surveillance applications such as object detection and scene understanding. In this paper, we present a novel pixel-to-model (P2M) paradigm for background modeling and restoration in surveillance scenes. In particular, the proposed approach models the background with a set of context features for each pixel, which are compressively sensed from local patches. We determine whether a pixel belongs to the background according to the minimum P2M distance, which measures the similarity between the pixel and its background model in the space of compressive local descriptors. The pixel feature descriptors of the background model are properly updated with respect to the minimum P2M distance. Meanwhile, the neighboring background model will be renewed according to the maximum P2M distance to handle ghost holes. The P2M distance plays an important role of background reliability in the 3-D spatial–temporal domain of surveillance videos, leading to the robust background model and recovered background videos. We applied the proposed P2M distance for foreground detection and background restoration on synthetic and real-world surveillance videos. Experimental results show that the proposed P2M approach outperforms the state-of-the-art approaches both in indoor and outdoor surveillance scenes. [ABSTRACT FROM PUBLISHER]

A novel superpixel extraction algorithm using a higher order energy optimization framework is proposed in this paper. We first adopt the $k$ -means clustering technique to quickly get an initial superpixel result. Then a higher order energy function is employed to optimize and refine these initial superpixels. We use a more general higher order energy function that includes a first-order data term, a second-order smoothness term, and a higher order term. The presegments are employed to provide the prior information of sufficient edges and segment regions for our higher order energy term. According to the texture measurement in different local regions, our algorithm adaptively computes the proper ratios of different energy terms to obtain a better superpixel performance. The experimental results demonstrate that our method using the higher order energy generates better results with well-aligned boundaries and homogeneous effects than the existing superpixel algorithms. [ABSTRACT FROM PUBLISHER]

Supervoxels have been widely used as a preprocessing step to exploit object boundaries to improve the performance of video processing tasks. However, most of the traditional supervoxel algorithms do not perform well in regions with complex textures or weak boundaries. These methods may generate supervoxels with overlapping boundaries. In this paper, we present the novel video supervoxel generation algorithm using partially absorbing random walks to get more accurate supervoxels in these regions. Our spatial–temporal framework is introduced by making full use of the appearance and motion cues, which effectively exploits the temporal consistency in video sequence. Moreover, we build a novel Laplacian optimization structure using two adjacent frames to make our approach more efficient. Experimental results demonstrated that our method achieved better performance than the state-of-the-art supervoxel algorithms. [ABSTRACT FROM PUBLISHER]

Sparse representation-based visual tracking approaches have attracted increasing interests in the community in recent years. The main idea is to linearly represent each target candidate using a set of target and trivial templates, while imposing a sparsity constraint onto the representation coefficients. After we obtain the coefficients using \ell 1 -norm minimization methods, the candidate with the lowest error, when it is reconstructed using only the target templates and the associated coefficients, is considered as the tracking result. In spite of promising system performance widely reported, it is unclear if the performance of these trackers can be maximized. In addition, computational complexity caused by the dimensionality of the feature space limits these algorithms in real-time applications. In this paper, we propose a real-time visual tracking method based on structurally random projection (RP) and weighted least squares (WLS) techniques. In particular, to enhance the discriminative capability of the tracker, we introduce background templates to the linear representation framework. To handle appearance variations over time, we relax the sparsity constraint using a WLS method to obtain the representation coefficients. To further reduce the computational complexity, structurally RP is used to reduce the dimensionality of the feature space, while preserving the pairwise distances between the data points in the feature space. Experimental results show that the proposed approach outperforms several state-of-the-art tracking methods. [ABSTRACT FROM AUTHOR]

With the rapid development of the intelligent video surveillance (IVS), person re-identification, which is a difficult yet unavoidable problem in video surveillance, has received increasing attention in recent years. That is because computer capacity has shown remarkable progress and the task of person re-identification plays a critical role in video surveillance systems. In short, person re-identification aims to find an individual again that has been observed over different cameras. It has been reported that KISS metric learning has obtained the state of the art performance for person re-identification on the VIPeR dataset refid="ref39"/. However, given a small size training set, the estimation to the inverse of a covariance matrix is not stable and thus the resulting performance can be poor. In this paper, we present regularized smoothing KISS metric learning (RS-KISS) by seamlessly integrating smoothing and regularization techniques for robustly estimating covariance matrices. RS-KISS is superior to KISS, because RS-KISS can enlarge the underestimated small eigenvalues and can reduce the overestimated large eigenvalues of the estimated covariance matrix in an effective way. By providing additional data, we can obtain a more robust model by RS-KISS. However, retraining RS-KISS on all the available examples in a straightforward way is time consuming, so we introduce incremental learning to RS-KISS. We thoroughly conduct experiments on the VIPeR dataset and verify that 1) RS-KISS completely beats all available results for person re-identification and 2) incremental RS-KISS performs as well as RS-KISS but reduces the computational cost significantly. [ABSTRACT FROM AUTHOR]

We propose a novel method to model human actions by explicitly coding motion and structure features that are separately extracted from video sequences. Firstly, the motion template (one feature map) is applied to encode the motion information and image planes (five feature maps) are extracted from the volume of differences of frames to capture the structure information. The Gaussian pyramid and center-surround operations are performed on each of the six obtained feature maps, decomposing each feature map into a set of subband maps. Biologically inspired features are then extracted by successively applying Gabor filtering and max pooling on each subband map. To make a compact representation, discriminative locality alignment is employed to embed the high-dimensional features into a low-dimensional manifold space. In contrast to sparse representations based on detected interest points, which suffer from the loss of structure information, the proposed model takes into account the motion and structure information simultaneously and integrates them in a unified framework; it therefore provides an informative and compact representation of human actions. The proposed method is evaluated on the KTH, the multiview IXMAS, and the challenging UCF sports datasets and outperforms state-of-the-art techniques on action recognition [ABSTRACT FROM AUTHOR]

Sketch–photo synthesis plays an important role in sketch-based face photo retrieval and photo-based face sketch retrieval systems. In this paper, we propose an automatic sketch–photo synthesis and retrieval algorithm based on sparse representation. The proposed sketch–photo synthesis method works at patch level and is composed of two steps: sparse neighbor selection (SNS) for an initial estimate of the pseudoimage (pseudosketch or pseudophoto) and sparse-representation-based enhancement (SRE) for further improving the quality of the synthesized image. SNS can find closely related neighbors adaptively and then generate an initial estimate for the pseudoimage. In SRE, a coupled sparse representation model is first constructed to learn the mapping between sketch patches and photo patches, and a patch-derivative-based sparse representation method is subsequently applied to enhance the quality of the synthesized photos and sketches. Finally, four retrieval modes, namely, sketch-based, photo-based, pseudosketch-based, and pseudophoto-based retrieval are proposed, and a retrieval algorithm is developed by using sparse representation. Extensive experimental results illustrate the effectiveness of the proposed face sketch–photo synthesis and retrieval algorithms. [ABSTRACT FROM AUTHOR]

One of the primary goals of the airborne vehicle detection system is to reduce the risks of incident collisions and to relieve traffic jam caused by the increasing number of vehicles. Different from the stationary systems, which are usually fixed on buildings, the airborne systems in unmanned aircrafts or satellites take the advantages of wider view angle and higher mobility. However, detecting vehicles in airborne videos is a challenging task because of the scene complexity and platform movement. The direct application of the traditional image processing techniques to the problem may result in low detection rate or cannot meet the requirements of real-time applications. To address these problems, a new and efficient method composed by two stages, attention focus extraction and vehicle classification is proposed in this paper. Our work makes two key contributions. The first is the introduction of a new attention focus extraction algorithm, which can quickly detect the candidate vehicle regions to make the algorithm focus on much smaller regions for faster computation. The second contribution is a simple and efficient classification process, which is built using the AdaBoost learning algorithm. The classification process, which is a hierarchical structure, is designed to obtain a lower false alarm rate by looking for vehicles in the candidate regions. Experimental results demonstrate that, compared with other representative algorithms, our method can obtain better performance in terms of higher detection rate and lower false positive rate, while meeting the needs of real-time application. [ABSTRACT FROM AUTHOR]

Visual surveillance from low-altitude airborne platforms plays a key role in urban traffic surveillance. Moving vehicle detection and motion analysis are very important for such a system. However, illumination variance, scene complexity, and platform motion make the tasks very challenging. In addition, the used algorithms have to be computationally efficient in order to be used on a real-time platform. To deal with these problems, a new framework for vehicle detection and motion analysis from low-altitude airborne videos is proposed. Our paper has two major contributions. First, to speed up feature extraction and to retain additional global features in different scales for higher classification accuracy, a boosting light and pyramid sampling histogram of oriented gradients feature extraction method is proposed. Second, to efficiently correlate vehicles across different frames for vehicle motion trajectories computation, a spatio-temporal appearance-related similarity measure is proposed. Compared to other representative existing methods, our experimental results showed that the proposed method is able to achieve better performance with higher detection rate, lower false positive rate, and faster detection speed. [ABSTRACT FROM AUTHOR]

Histogram-based lossless data embedding (LDE) has been recognized as an effective and efficient way for copyright protection of multimedia. Recently, a LDE method using the statistical quantity histogram has achieved good performance, which utilizes the similarity of the arithmetic average of difference histogram (AADH) to reduce the diversity of images and ensure the stable performance of LDE. However, this method is strongly dependent on some assumptions, which limits its applications in practice. In addition, the capacities of the images with the flat AADH, e.g., texture images, are a little bit low. For this purpose, we develop a novel framework for LDE by incorporating the merits from the generalized statistical quantity histogram (GSQH) and the histogram-based embedding. Algorithmically, we design the GSQH driven LDE framework carefully so that it: 1) utilizes the similarity and sparsity of GSQH to construct an efficient embedding carrier, leading to a general and stable framework; 2) is widely adaptable for different kinds of images, due to the usage of the divide-and-conquer strategy; 3) is scalable for different capacity requirements and avoids the capacity problems caused by the flat histogram distribution; 4) is conditionally robust against JPEG compression under a suitable scale factor; and 5) is secure for copyright protection because of the safe storage and transmission of side information. Thorough experiments over three kinds of images demonstrate the effectiveness of the proposed framework. [ABSTRACT FROM AUTHOR]