Search

With the proliferation of video data, video summarization is an ideal tool for users to browse video content rapidly. In this paper, we propose a novel foveated convolutional neural networks for dynamic video summarization. We are the first to integrate gaze information into a deep learning network for video summarization. Foveated images are constructed based on subjects’ eye movements to represent the spatial information of the input video. Multi-frame motion vectors are stacked across several adjacent frames to convey the motion clues. To evaluate the proposed method, experiments are conducted on two video summarization benchmark datasets. The experimental results validate the effectiveness of the gaze information for video summarization despite the fact that the eye movements are collected from different subjects from those who generated summaries. Empirical validations also demonstrate that our proposed foveated convolutional neural networks for video summarization can achieve state-of-the-art performances on these benchmark datasets.

Saliency cuts aims to segment salient objects from a given saliency map. The existing saliency cuts methods are fixed to the input cues. It limits their performance when the input cues are changed. In this paper, we propose a novel saliency cuts method named adaptive saliency cuts, which takes advantage of all the input cues in a unified framework and adjusts its components adaptively. Given a saliency map, we first generate segmentation seeds with adaptive triple thresholding. Next, we extend GrabCut by combining different input cues, and use it to generate a rough-labeled map of salient objects. Finally, we refine the boundaries of the salient objects with adaptive initialized segmentation, and produce an accurate binary mask. To the best of our knowledge, this method is the first adaptive saliency cuts method for different input cues. We validated the proposed method on MSRA10K and NJU2000. The experimental results demonstrate that our method outperforms the state-of-the-art methods.

Object proposal aims to locate category-independent objects in a given image with a limited number of object candidates indicated by bounding boxes, which can be served as a fundamental of various multimedia applications. Current evaluation criteria based on recall cannot reveal the real abilities of different object proposal methods in objectness measurement. In this paper, we propose a novel object proposal evaluation criterion instead of recall, named objectness measurement ability (OMA). We first analyze the probability to hit an object by non-repetitive random sampling (HPRS), and provide an algorithm for calculating HPRS efficiently. Based on HPRS, we define OMA and extend three commonly used object proposal evaluation criteria by replacing recall with OMA. We evaluated six typical object proposal methods using recall based criteria and OMA based criteria on the test data of PASCAL VOC 2007 and PASCAL VOC 2012. The experimental results show that OMA based criteria can provide more stable evaluation results than recall based ones in revealing objectness measurement ability.

Image steganalysis is to discriminate innocent images and those suspected images with hidden messages. This task is very challenging for modern adaptive steganography, since modifications due to message hiding are extremely small. Recent studies show that Convolutional Neural Networks (CNN) have demonstrated superior performances than traditional steganalytic methods. Following this idea, we propose a novel CNN model for image steganalysis based on residual learning. The proposed Deep Residual learning based Network (DRN) shows two attractive properties than existing CNN based methods. First, the model usually contains a large number of network layers, which proves to be effective to capture the complex statistics of digital images. Second, the residual learning in DRN preserves the stego signal coming from secret messages, which is extremely beneficial for the discrimination of cover images and stego images. Comprehensive experiments on standard dataset show that the DRN model can detect the state of arts steganographic algorithms at a high accuracy. It also outperforms the classical rich model method and several recently proposed CNN based methods.

Static video summarization is recognized as an effective way for users to quickly browse and comprehend large numbers of videos. In this paper, we formulate static video summarization as a clustering problem. Inspired by the idea from high density peaks search clustering algorithm, we propose an effective clustering algorithm by integrating important properties of video to gather similar frames into clusters. Finally, all clusters' center will be collected as static video summarization. Compared with existing clustering-based video summarization approaches, our work can detect frames which are highly relevant and generate representative clusters automatically. We evaluate our proposed work by comparing it with several state-of-the-art clustering-based video summarization methods and some classical clustering algorithms. The experimental results evidence that our proposed method has better performance and efficiency.