Search

The rise in the availability of video content for access via the Internet and the medium of television has resulted in the development of automatic search procedures to retrieve the desired video. Searches can be simplified and hastened by employing automatic classification of videos. This paper proposes a descriptor called the Spatio-Temporal Histogram of Radon Projections (STHRP) for representing the temporal pattern of the contents of a video and demonstrates its application to video classification and retrieval. The first step in STHRP pattern computation is to represent any video as Three Orthogonal Planes (TOPs), i.e., XY, XT and YT, signifying the spatial and temporal contents. Frames corresponding to each plane are partitioned into overlapping blocks. Radon projections are obtained over these blocks at different orientations, resulting in weighted transform coefficients that are normalized and grouped into bins. Linear Discriminant Analysis (LDA) is performed over these coefficients of the TOPs to arrive at a compact description of STHRP pattern. Compared to existing classification and retrieval approaches, the proposed descriptor is highly robust to translation, rotation and illumination variations in videos. To evaluate the capabilities of the invariant STHRP pattern, we analyse the performance by conducting experiments on the UCF-101, HMDB51, 10contexts and TRECVID data sets for classification and retrieval using a bagged tree model. Experimental evaluation of video classification reveals that STHRP pattern can achieve classification rates of 96.15%, 71.7%, 93.24% and 97.3% for the UCF-101, HMDB51,10contexts and TRECVID 2005 data sets respectively. We conducted retrieval experiments on the TRECVID 2005, JHMDB and 10contexts data sets and the results revealed that STHRP pattern is able to provide the videos relevant to the user's query in minimal time (0.05s) with a good precision rate.

Video shot boundary detection (VSBD) is one of the most essential criteria for many intelligent video analysis-related applications, such as video retrieval, indexing, browsing, categorization and summarization. VSBD aims to segment big video data into meaningful fragments known as shots. This paper put forwards a new pyramidal opponent colour-shape (POCS) model which can detect abrupt transition (AT) and gradual transition (GT) simultaneously, even in the presence of illumination changes, huge object movement between frames, and fast camera motion. First, the content of frames in the video subjected to VSBD is represented by the proposed POCS model. Consequently, the temporal nature of the POCS model is subjected to a suitable segment (SS) selection procedure in order to minimize the complexity of VSBD method. The SS from the video frames is examined for transitions within it using a bagged-trees classifier (BTC) learned on a balanced training set via parallel processing. To prove the superiority of the proposed VSBD algorithm, it is evaluated on the TRECVID 2001, TRECVID2007 and VIDEOSEG2004 data sets for classifying the basic units of video according to no transition (NT), AT and GT. The experimental evaluation results in an F1-score of 95.13%, 98.13% and 97.11% on the TRECVID 2001, TRECVID2007 and VIDEOSEG2004 data sets, respectively.

Video indexing and retrieval play a vital role in video management systems involving modules such as shot boundary detection, key frame extraction, and feature representation. Among these modules, Video Shot Boundary Detection (VSBD) is the important primary step, over which the entire retrieval performance relies on. In this paper, a new shot boundary detection mechanism based on context-driven saliency map is proposed to detect the transitions in the highly challenging videos with varying lighting effects, object and camera motion. This saliency map detects the salient regions in the frame along with the vital background scenes. The statistical features derived from the saliency map are used to derive the dissimilarity value. The dissimilarity value is compared to the threshold to determine the location and types of transition in the video. To evaluate the proposed framework for VSBD, the benchmark dataset namely TRECVID is used. It is inferred that this saliency-based VSBD approach yields promising results when evaluated on TRECVID and IDV dataset. The average F-Score in detecting the overall transitions on TRECVID dataset is 93.16% and IDV Dataset is 92.23%.

Video shot boundary detection is the primary task for content based video management and retrieval system. This paper proposes a shot boundary detection strategy by exploiting the pros of Normalized Periodogram for efficiently representing the content of the video. A Normalized Periodogram based distance metric to detect the key frames using shot boundary, namely Distance- Left-Right (DLR), is addressed, which is computed on a sliding sub-window basis. The DLR sequence is used to detect the suspected shot boundary frames and a transition type detection procedure is adapted to these suspected frames for discriminating the abrupt and gradual transitions. The proposed shot boundary detection methodology yields Precision—95.02%, Recall—93.15% and F1 score—94.07% for cut, Precision—86.57%, Recall—86.67% and F1 score—86.61% for gradual, Precision—90.6%, Recall—90.02% and F1 score—90.3% for overall transitions. Experimental results show that the proposed approach is superior to the recently available shot boundary detection techniques because of its robustness and simplicity, and presents an effective distance metric to detect the shot boundary.

Video summarization and retrieval plays a vital role in video management systems involving modules such as shot boundary detection, key frame extraction and feature representation. Among these modules, shot boundary detection (SBD) is the important primary step, over which the entire retrieval performance relies on. In this paper, a new hierarchical shot boundary detection mechanism is proposed to alleviate the redundant key frames in video summarization. Initially, the abrupt transitions in the videos are detected using Weber Local Descriptor (WLD) and primary key frames are extracted. After primary key frame extraction, Fourier Transform (FT) which is complementary to WLD is used to prune the redundant key frames on the highly challenging videos with varying lighting effects, object and camera motion. The proposed hierarchical SBD algorithm is evaluated on three datasets viz. TRECVID, OPEN VIDEO. The validation of the proposed algorithm on these dataset yields promising results in terms of precision, recall, F-score, and Compression rate.

One of the promising new directions for Content Based Video Retrieval is object based retrieval which allows the user to manipulate video object as a part of searching and browsing. The major obstacle for the use of objects in video retrieval is the appropriate representation of objects in video database. The purpose of this work is to present an object based framework consisting of entire processing chain needed to segment, represent and retrieve video object via appropriate content extraction. Such an object based approach can reduce the semantic gap to an acceptable level and provide satisfactory results in terms of performance metrics like Precision, Recall, Specificity, Sensitivity and Error rate. With this aim, Histogram of Fourier Coefficients (HFC) is proposed to represent the objects in video frames. In addition, the performance of the video retrieval system based on the proposed HFC is validated using Classification and Regression Trees, Random Forest and Extreme Learning Machine model. Experimental results prove that HFC is an efficient content descriptor for high performance video retrieval.

Detecting what attracts human attention is one of the vital tasks for visual processing. Saliency detection finds out the location of foci of attention on an outstanding object in image/video sequences. However, temporal information in videos play major role in human visual perception in locating salient objects. This paper presents a novel approach to detect salient object in a video using spatio-temporal textural saliency which also includes temporal information, an important aspect in videos. In this work, the context driven static saliency extracted from Lab color space in XY plane is combined with the local phase quantization on three orthogonal planes (LPQ-TOP) driven dynamic saliency to detect the spatio-temporal saliency in videos. The dynamic saliency is obtained by fusing two temporal saliencies extracted from XT-plane and YT-plane using LPQ texture feature, which extracts the temporal salient region. This approach is evaluated on Benchmark dataset and the result shows that the proposed saliency approach yields promising performance.

Video object segmentation, a binary labelling problem is vital in various applications including object tracking, action recognition, video summarization, video editing, object based encoding and video retrieval. This paper presents an overview of recent strategies in video object segmentation, focusing on the techniques for solving challenges like complex and moving background, illumination changes, occlusions, motion blur, shadow effect and view point variation. Significant works evolved in this research field over recent years are categorized based on the challenges solved by the researchers. A list of challenging datasets and evaluation metrics available for video object segmentation is presented. Finally, research gaps in this domain are discussed.