Search

A generative adversarial network (GAN) is a framework composed of a generator and a discriminator. GAN learns the deep attributes without the need for hugely annotated training data. This learning is attained by back-propagation approach through competition between generator and discriminator network. Since 2014, GAN has been used in a wide variety of applications such as game development, security, image editing, data generation, attention prediction, and so on. In this article, we explore the fundamental working principle, architecture of GAN, and the effectiveness of GAN for video analytics. Video analytics has become a crucial research area for the academic world and the industry thanks to the availability of IP CCTV as well as the progressive growth in the video content analysis algorithms. Automation in video analytics has come into practice due to the extensive growth of deep learning algorithm. The prime intention of this chapter is to provide a comprehensive review of the GAN variants available for video analytics, explore different architectures, and further the techniques used in those respective applications.

Image Retrieval (IR) framework enables the user to search query images in order to retrieve images stored in the database according to their advantage. Content Based Image Retrieval (CBIR) is a technique which uses visual features of an image such as color, shape and texture feature to retrieve images of user interest. The major issue that exists in any CBIR system is semantic gap and computational time. Hence this work aims to provide an exchange off between computational time and accuracy. To extract the color, texture and shape feature, the RGB color histogram from the three independent color channels, Local Binary Pattern (LBP) from the gray scale image Histogram of oriented gradients are derived respectively. These three features are concatenated to obtain the feature vector of the images in the database. The dimensionality of the feature vector is reduced by Linear Discriminant Analysis (LDA). The compact feature vector set is trained using Random Vector Functional Link (RVFL) network to create the knowledge base. In the testing phase, once the user rises a query, the query feature vector is derived for the corresponding query image and tested against the knowledge base using RVFL classifier. Using the class code obtained by RVFL classifier, the images are retrieved using Minkowski distance. The performance of the proposed algorithm is validated by evaluating it on a Corel Image database using metrics like precision, Recall and F-score. The proposed feature combination along with LDA and RVFL provides better results in retrieving the query image.

The rise in the availability of video content for access via the Internet and the medium of television has resulted in the development of automatic search procedures to retrieve the desired video. Searches can be simplified and hastened by employing automatic classification of videos. This paper proposes a descriptor called the Spatio-Temporal Histogram of Radon Projections (STHRP) for representing the temporal pattern of the contents of a video and demonstrates its application to video classification and retrieval. The first step in STHRP pattern computation is to represent any video as Three Orthogonal Planes (TOPs), i.e., XY, XT and YT, signifying the spatial and temporal contents. Frames corresponding to each plane are partitioned into overlapping blocks. Radon projections are obtained over these blocks at different orientations, resulting in weighted transform coefficients that are normalized and grouped into bins. Linear Discriminant Analysis (LDA) is performed over these coefficients of the TOPs to arrive at a compact description of STHRP pattern. Compared to existing classification and retrieval approaches, the proposed descriptor is highly robust to translation, rotation and illumination variations in videos. To evaluate the capabilities of the invariant STHRP pattern, we analyse the performance by conducting experiments on the UCF-101, HMDB51, 10contexts and TRECVID data sets for classification and retrieval using a bagged tree model. Experimental evaluation of video classification reveals that STHRP pattern can achieve classification rates of 96.15%, 71.7%, 93.24% and 97.3% for the UCF-101, HMDB51,10contexts and TRECVID 2005 data sets respectively. We conducted retrieval experiments on the TRECVID 2005, JHMDB and 10contexts data sets and the results revealed that STHRP pattern is able to provide the videos relevant to the user's query in minimal time (0.05s) with a good precision rate.

Video shot boundary detection (VSBD) is one of the most essential criteria for many intelligent video analysis-related applications, such as video retrieval, indexing, browsing, categorization and summarization. VSBD aims to segment big video data into meaningful fragments known as shots. This paper put forwards a new pyramidal opponent colour-shape (POCS) model which can detect abrupt transition (AT) and gradual transition (GT) simultaneously, even in the presence of illumination changes, huge object movement between frames, and fast camera motion. First, the content of frames in the video subjected to VSBD is represented by the proposed POCS model. Consequently, the temporal nature of the POCS model is subjected to a suitable segment (SS) selection procedure in order to minimize the complexity of VSBD method. The SS from the video frames is examined for transitions within it using a bagged-trees classifier (BTC) learned on a balanced training set via parallel processing. To prove the superiority of the proposed VSBD algorithm, it is evaluated on the TRECVID 2001, TRECVID2007 and VIDEOSEG2004 data sets for classifying the basic units of video according to no transition (NT), AT and GT. The experimental evaluation results in an F1-score of 95.13%, 98.13% and 97.11% on the TRECVID 2001, TRECVID2007 and VIDEOSEG2004 data sets, respectively.

Video indexing and retrieval play a vital role in video management systems involving modules such as shot boundary detection, key frame extraction, and feature representation. Among these modules, Video Shot Boundary Detection (VSBD) is the important primary step, over which the entire retrieval performance relies on. In this paper, a new shot boundary detection mechanism based on context-driven saliency map is proposed to detect the transitions in the highly challenging videos with varying lighting effects, object and camera motion. This saliency map detects the salient regions in the frame along with the vital background scenes. The statistical features derived from the saliency map are used to derive the dissimilarity value. The dissimilarity value is compared to the threshold to determine the location and types of transition in the video. To evaluate the proposed framework for VSBD, the benchmark dataset namely TRECVID is used. It is inferred that this saliency-based VSBD approach yields promising results when evaluated on TRECVID and IDV dataset. The average F-Score in detecting the overall transitions on TRECVID dataset is 93.16% and IDV Dataset is 92.23%.

Video shot boundary detection is the primary task for content based video management and retrieval system. This paper proposes a shot boundary detection strategy by exploiting the pros of Normalized Periodogram for efficiently representing the content of the video. A Normalized Periodogram based distance metric to detect the key frames using shot boundary, namely Distance- Left-Right (DLR), is addressed, which is computed on a sliding sub-window basis. The DLR sequence is used to detect the suspected shot boundary frames and a transition type detection procedure is adapted to these suspected frames for discriminating the abrupt and gradual transitions. The proposed shot boundary detection methodology yields Precision—95.02%, Recall—93.15% and F1 score—94.07% for cut, Precision—86.57%, Recall—86.67% and F1 score—86.61% for gradual, Precision—90.6%, Recall—90.02% and F1 score—90.3% for overall transitions. Experimental results show that the proposed approach is superior to the recently available shot boundary detection techniques because of its robustness and simplicity, and presents an effective distance metric to detect the shot boundary.

Video summarization and retrieval plays a vital role in video management systems involving modules such as shot boundary detection, key frame extraction and feature representation. Among these modules, shot boundary detection (SBD) is the important primary step, over which the entire retrieval performance relies on. In this paper, a new hierarchical shot boundary detection mechanism is proposed to alleviate the redundant key frames in video summarization. Initially, the abrupt transitions in the videos are detected using Weber Local Descriptor (WLD) and primary key frames are extracted. After primary key frame extraction, Fourier Transform (FT) which is complementary to WLD is used to prune the redundant key frames on the highly challenging videos with varying lighting effects, object and camera motion. The proposed hierarchical SBD algorithm is evaluated on three datasets viz. TRECVID, OPEN VIDEO. The validation of the proposed algorithm on these dataset yields promising results in terms of precision, recall, F-score, and Compression rate.

One of the promising new directions for Content Based Video Retrieval is object based retrieval which allows the user to manipulate video object as a part of searching and browsing. The major obstacle for the use of objects in video retrieval is the appropriate representation of objects in video database. The purpose of this work is to present an object based framework consisting of entire processing chain needed to segment, represent and retrieve video object via appropriate content extraction. Such an object based approach can reduce the semantic gap to an acceptable level and provide satisfactory results in terms of performance metrics like Precision, Recall, Specificity, Sensitivity and Error rate. With this aim, Histogram of Fourier Coefficients (HFC) is proposed to represent the objects in video frames. In addition, the performance of the video retrieval system based on the proposed HFC is validated using Classification and Regression Trees, Random Forest and Extreme Learning Machine model. Experimental results prove that HFC is an efficient content descriptor for high performance video retrieval.

Detecting what attracts human attention is one of the vital tasks for visual processing. Saliency detection finds out the location of foci of attention on an outstanding object in image/video sequences. However, temporal information in videos play major role in human visual perception in locating salient objects. This paper presents a novel approach to detect salient object in a video using spatio-temporal textural saliency which also includes temporal information, an important aspect in videos. In this work, the context driven static saliency extracted from Lab color space in XY plane is combined with the local phase quantization on three orthogonal planes (LPQ-TOP) driven dynamic saliency to detect the spatio-temporal saliency in videos. The dynamic saliency is obtained by fusing two temporal saliencies extracted from XT-plane and YT-plane using LPQ texture feature, which extracts the temporal salient region. This approach is evaluated on Benchmark dataset and the result shows that the proposed saliency approach yields promising performance.

Video object segmentation, a binary labelling problem is vital in various applications including object tracking, action recognition, video summarization, video editing, object based encoding and video retrieval. This paper presents an overview of recent strategies in video object segmentation, focusing on the techniques for solving challenges like complex and moving background, illumination changes, occlusions, motion blur, shadow effect and view point variation. Significant works evolved in this research field over recent years are categorized based on the challenges solved by the researchers. A list of challenging datasets and evaluation metrics available for video object segmentation is presented. Finally, research gaps in this domain are discussed.

Lightning is a serious natural calamity all over the world, kills people and wild species, damages human shelters, induces huge forest fire and crashes outdoor camera. Mainly in hilly regions, the purpose of the lightning arrestor fails, seeking vision based solution to detect the lightning in such areas. This paper addresses the detection of lightning in captured videos as natural abruption detection in video frames. The proposed lightning detection technique models the luminance channel of frames in the video as Gray Level Co-occurrence Matrix (GLCM) which defines the arrangement of regions within a frame through statistics. Among various attributes derived from GLCM, contrast and homogeneity which plays vital role in detecting the variations (lightning) between consecutive frames is derived from the GLCM frames constituting the frame feature vector. The difference signal based on the frame feature vector is constructed followed by statistical thresholding which detects the lightning affected frames in the captured video. Experiments on the user collected videos show the high speed and astounding performance of the proposed lightning detection scheme compared to Saturation channel.

A finished product should be quality assured to meet the needs, expectations and requirements it is intended for. As quality through human inspection may lead to errors and may not be adequate for instant and mass production, current high speed production lines require automated inspection with precise critical objective measurements and high reliability. Image processing has become a key tool in the area of automated vision inspection for commercial marketability of finished goods utilizing devices for optical, non contact sensing to automatically receive and interpret an image of the product for integrity checking, correct labelling, surface finish, quality control, etc and has been used in many industries. In the soap industry, the company logo is engraved on the surface of the bar soap to a particular uniform depth which needs to be inspected against excess soap removal, non uniform engraving and absence of engraving. These defects of engraving lead to customer's displeasure and thus a loss of business to the manufacturer. This paper presents an automated system using machine vision and image processing techniques to compute the depth at the engravement and to verify whether it meets the preset standards.

Many practical applications such as medical image segmentation, object detection, recognition tasks and video surveillance have need for accurate image segmentation techniques. Hence image segmentation is an important technique for image processing which is regarded as first step for image analysis. In this paper an image segmentation technique based on Bacterial Foraging (BF) and Particle Swarm Optimization (PSO) algorithm is addressed. Initially adaptation is done on BF algorithm by computing the step length using the number of variables in the search space. Further, on exhaustive analysis of BF algorithm, it was revealed that the tumble behavior will lead to random delay in searching optimal solutions and premature convergence. This synergy algorithm makes use of PSO in providing social information and adaptive BF algorithm in finding new optimal threshold values using elimination and dispersal. The proposed method has been applied to few benchmark images with promising results.