Search

In this paper, we propose an algorithm of anomaly detection in crowded scenes by using sparse representation over the normal bases. First, the histogram of maximal optical flow projection (HMOFP) features are extracted from a set of normal training data. Then, the online dictionary learning algorithm is used to train an optimal dictionary with proper redundancy, which is better than the dictionary simply composed by the HMOFP features of the whole training data. In order to detect the normalness of a frame, the l -norm of the sparse reconstruction coefficients is used as the Reconstruction Coefficient Sparsity (RCS). Our algorithm is effective for both global abnormal events (GAE) and local abnormal events (LAE). We evaluate our method on three benchmark datasets-the UMN dataset, the PETS2009 dataset and the UCSD Ped1 dataset. Compared with the most popular methods, experimental results show that our algorithm achieves good results especially for the pixel-level local abnormal event localization. [ABSTRACT FROM AUTHOR]

Image retrieval is a recognition technique in the field of computer vision. In most cases, high-quality retrieval is often supported by adequate learning instances. However, in the process of learning instance selection, some useless, repeated, invalid, and even mistaken learning instances are often selected. Low-quality instances not only add to the computing burden but also decrease the retrieval quality. In this study, we propose a learning instance optimization method. Initially, we classify the images into scene and object images by using the K-means clustering model. We use different methods to handle these two groups of images. For scene images, we use the Euclidean distance of the GIST descriptor to select the optimized learning instances. For object images, we use the improved spatial pyramid matching and optimal instance distance methods to select the optimized learning instances. Finally, we implement experiments using one large image database to check the effectiveness of our proposed algorithm. Results show that our method can not only improve retrieval quality but also decrease the number of learning instances. [ABSTRACT FROM AUTHOR]

Spatial support is an effective method of improving object recognition that is widely used in the field of computer vision. Compared to various other spatial support methods, such as sliding windows, spatial support based on image segmentation is a classic technique with high-quality segmentation that can positively contribute to image recognition. However, over-segmentation and under-segmentation often occur in the segmentation process. It is difficult for object classifiers to recognize objects in low-quality, segmented images, thus low-quality segmentation will reduce recognition accuracy. In order to resolve this drawback, watershed segmentation, multi-scale decompositions based on WLS (Weighted-Least Squares) filters and multi-layer smoothing, have been used to process the images. This methodology was utilized to maintain the sharp regions' boundaries, while smoothing regions which could contribute to accurately segmenting the images. After obtaining high-quality segmentation, the images could then be utilized in image recognition. The superiority of this paper's methodology compared to that of previous methods has been demonstrated herein. Experiments using a large database have demonstrated that this methodology is capable of improving image recognition through optimizing segmentation results. [ABSTRACT FROM AUTHOR]

This paper discusses the task of human action recognition. This task is important to applications like video surveillance and video retrieval. Most of the existing local interest points based works on human action analysis, lost the information about spatio-temporal distribution of features and neglected the relationship between features and each defined actions. In this paper, through the analysis of feature distribution and their interactions over spatio-temporal domain, we propose a novel projection transform to take the two factors into account. A video sequence of human action in our perspective can be modeled by three types of features of spatio-temporal interest points: the global projection transform feature, the relative position distribution feature and the bag of visual words based feature. Then a new context-K-nearest-neighbor classifier is utilized to fuse them to form discriminative feature sets for action matching. In most of the case, our experiments have indicated that the novel method outperforms other previous published results on the Weizmann and KTH datasets. [ABSTRACT FROM AUTHOR]

As an essentially multi-label classification problem, audio concept detection is normally solved by treating concepts independently. Since in this process the original useful concept correlation information is missing, this paper proposes a new model named Correlated-Aspect Gaussian Mixture Model (C-AGMM) to take advantage of such a clue for enhancing multi-label audio concept detection. Originating from Aspect Gaussian Mixture Model (AGMM) which improves GMM by incorporating it into probabilistic Latent Semantic Analysis (pLSA), C-AGMM still learns a probabilistic model of the whole audio clip by regarding concepts as its component elements. However, different from AGMM that assumes concepts independent with each other, C-AGMM considers their distribution on a sub-manifold embedded in the ambient space. With an assumption that if two concepts are close in the intrinsic geometry of this distribution then their conditional probability distributions are likely to show similarity, a graph regularizer is exploited to model the correlation between these concepts. Following the Maximum Likelihood Estimate principle, model parameters of C-AGMM encoding the concept correlation clue are derived and used directly as the detection criterion. Experiments on two datasets show the effectiveness of our proposed model. [ABSTRACT FROM AUTHOR]

In this paper, we present a new object recognition model with few instances based on semantic distance. Learning objects with many instances have been studied in computer vision for many years. However, in many cases, not enough positive instances occur, especially for some special categories. We must take full advantage of all instances, including those that do not belong to the category. The main insight is that, given a few positive instances from one category, we can define some other candidate instances as positive instances based on semantic distance to learn this model. Our model responds more strongly to instances with closer semantic distance to positive instances than to instances with farther semantic distance to positive instances. We use a regularized kernel machine algorithm to train the images from the database. The superiority of our method to existing object recognition methods is demonstrated. Experiments using an image database show that our method not only reduces the number of learning instances but also keeps the accurate rate of recognition. [ABSTRACT FROM AUTHOR]

As standard probabilistic latent semantic analysis (pLSA) is oriented to discrete quantity only, pLSA with Gaussian mixtures (GM-pLSA) succeeding in transferring it to continuous feature space is proposed, which uses Gaussian mixture model to describe the feature distribution under each latent aspect. However, inheriting from pLSA, GM-pLSA still overlooks the intrinsic interdependence between terms, which indeed is an important clue for performance improvement. In this paper, we present a graph regularized GM-pLSA (GRGM-pLSA) model as an extension of GM-pLSA to embed this term correlation information into the process of model learning. Specifically, grounded on the manifold regularization principle, a graph regularizer is introduced to characterize the correlation between terms; by imposing it on the objective function of GM-pLSA, model parameters of GRGM-pLSA are derived via corresponding expectation maximization algorithm. Furthermore, two applications to video content analysis are devised. One is video categorization where GRGM-pLSA serves for feature mapping with two kinds of sub-shot correlations, respectively, incorporated, while the other provides a new perspective on video concept detection, which transforms the detection task to a GRGM-pLSA-based visual-to-textual feature conversion problem. Extensive experiments and comparison with GM-pLSA and several state-of-the-art approaches in both applications demonstrate the effectiveness of GRGM-pLSA. [ABSTRACT FROM AUTHOR]

Low-resolution face recognition (LR FR) aims to recognize faces from small size or poor quality images with varying pose, illumination, expression, etc. It has received much attention with increasing demands for long distance surveillance applications, and extensive efforts have been made on LR FR research in recent years. However, many issues in LR FR are still unsolved, such as super-resolution (SR) for face recognition, resolution-robust features, unified feature spaces, and face detection at a distance, although many methods have been developed for that. This paper provides a comprehensive survey on these methods and discusses many related issues. First, it gives an overview on LR FR, including concept description, system architecture, and method categorization. Second, many representative methods are broadly reviewed and discussed. They are classified into two different categories, super-resolution for LR FR and resolution-robust feature representation for LR FR. Their strategies and advantages/disadvantages are elaborated. Some relevant issues such as databases and evaluations for LR FR are also presented. By generalizing their performances and limitations, promising trends and crucial issues for future research are finally discussed. [ABSTRACT FROM AUTHOR]

In this paper, we provide a unified expression to obtain the conditions on the restricted isometry constant δ(Φ). These conditions cover the important results proposed by Candes et al. and each of them is a sufficient condition for sparse signal recovery. In the noiseless case, when δ(Φ) satisfies any one of these conditions, the s-sparse signal can be exactly recovered via ( l) constrained minimization. [ABSTRACT FROM AUTHOR]

This paper discusses the task of continuous human action recognition. By continuous, it refers to videos that contain multiple actions which are connected together. This task is important to applications like video surveillance and content based video retrieval. It aims to identify the action category and detect the start and end key frame of each action. It is a challenging task due to the frequent changes of human actions and the ambiguity of action boundaries. In this paper, a novel and efficient continuous action recognition framework is proposed. Our approach is based on the bag of words representation. A visual local pattern is regarded as a word and the action is modeled by the distribution of words. A generative translation and scale invariant probabilistic Latent Semantic Analysis model is presented. The continuous action recognition result is obtained frame by frame and updated from time to time. Experimental results show that this approach is effective and efficient to recognize both isolated actions and continuous actions. [ABSTRACT FROM AUTHOR]

In this paper, a novel video matting method based on opacity propagation is proposed. It adopts a two-step framework by first segmenting each frame into bilayer and generating trimaps, and then applying matting algorithm to extract the final alpha mattes. In the bilayer segmentation step, an opacity propagation algorithm is used to predict the foreground object in the following frame. It propagates the opacity values of the former frame to the current, and generates an Opacity Map (OM), which is in most cases much more accurate than the previously used Probability Map (PM). OM is used to extract the foreground object by combining it with the graph cut algorithm. And then an accurate trimap is generated by a trimap refinement method based on a Local Gaussian Mixture Model (LGMM). At last, the opacity propagation is again applied to the unknown regions of the trimap to generate the accurate alpha mattes in a temporally coherent way. The experiment results demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Moving object segmentation is one of the most challenging issues in computer vision. In this paper, we propose a new algorithm for static camera foreground segmentation. It combines Gaussian mixture model (GMM) and active contours method, and produces much better results than conventional background subtraction methods. It formulates foreground segmentation as an energy minimization problem and minimizes the energy function using curve evolution method. Our algorithm integrates the GMM background model, shadow elimination term and curve evolution edge stopping term into energy function. It achieves more accurate segmentation than existing methods of the same type. Promising results on real images demonstrate the potential of the presented method. [ABSTRACT FROM AUTHOR]

Currently, many vision-based motion capture systems require passive markers attached to key locations on the human body. However, such systems are intrusive with limited application. The algorithm that we use for human motion capture in this paper is based on Markov random field (MRF) and dynamic graph cuts. It takes full account of the impact of 3D reconstruction error and integrates human motion capture and 3D reconstruction into MRF-MAP framework. For more accurate and robust performance, we extend our algorithm by incorporating color constraints into the pose estimation process. The advantages of incorporating color constraints are demonstrated by experimental results on several video sequences. [ABSTRACT FROM AUTHOR]

The video captioning task is generating description sentences by learning semantic information. It has a wide range of applications in areas such as video retrieval, automatic generation of subtitles and blind assistance. Visual semantic information plays a decisive role in video captioning. However, traditional methods are relatively rough for video feature modeling, failing to harness local and global features to understand temporal and spatial relationships. In this paper, we propose a video captioning model based on the Transformer and GCN network called “Relation-Enhanced Spatial–Temporal Hierarchical Transformer” (RESTHT). To address the above issues, we present a spatial–temporal hierarchical network framework to jointly model local and global features in terms of both time and space. For temporal modeling, our model learns the direct interactions between diverse video features and sentence features in the temporal sequence via the pre-trained GPT2, and the global feature construction encourages it to capture essential and relevant information. For spatial modeling, we use self-attention and GCN networks to learn the spatial relationship from appearance and motion perspectives jointly. Through spatial–temporal modeling, our method can comprehend the global time–space relationships of complex events in videos and catch the interaction between different objects to generate more accurate descriptions applicable to universal video captioning tasks. We conducted experiments on two widely used datasets, and especially in the MSVD dataset, our model improves the score of CIDEr by 6.1 compared to the baseline and excels present methods by 13. The results verify that our model can fully model the temporal and spatial relationship and outperforms other related models. [ABSTRACT FROM AUTHOR]