Search

Motion video recognition has been well explored in applications of computer vision. In this paper, we propose a novel video representation, which enhances motion recognition in videos based on SURF (Speeded-Up Robust Features) and two filters. Firstly, the detector scheme of SURF is used to detect the candidate points of the video because it is an efficient faster local feature detector. Secondly, by using the optical flow field and trajectory, the feature points can be filtered from the candidate points, which enables a robust and efficient extraction of motion feature points. Additionally, we introduce a descriptor, called MoSURF (Motion Speeded-Up Robust Features), based on SURF (Speeded-Up Robust Features), HOG (Histogram of Oriented Gradient), HOF (Histograms of Optical Flow), MBH(Motion Boundary Histograms), and trajectory information, which can effectively describe motion information and are complementary to each other. We evaluate our video representation under action classification on three motion video datasets namely KTH, YouTube, and UCF50. Compared with state-of-the-art methods, the proposed method shows advanced results on all datasets. [ABSTRACT FROM AUTHOR]

In this paper, we devise a saliency prediction model for stereoscopic videos that learns to explore saliency inspired by the component-based interactions including spatial, temporal, as well as depth cues. The model first takes advantage of specific structure of 3D residual network (3D-ResNet) to model the saliency driven by spatio-temporal coherence from consecutive frames. Subsequently, the saliency inferred by implicit-depth is automatically derived based on the displacement correlation between left and right views by leveraging a deep convolutional network (ConvNet). Finally, a component-wise refinement network is devised to produce final saliency maps over time by aggregating saliency distributions obtained from multiple components. In order to further facilitate research towards stereoscopic video saliency, we create a new dataset including 175 stereoscopic video sequences with diverse content, as well as their dense eye fixation annotations. Extensive experiments support that our proposed model can achieve superior performance compared to the state-of-the-art methods on all publicly available eye fixation datasets. [ABSTRACT FROM AUTHOR]

Human eye movements indicate important spatial information in static images and videos. However, videos contain additional temporal information and convey a storyline. We explore whether eye movement patterns reflect frame importance during video viewing and facilitate video summarization. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we present a novel method for content adaptation and video summarization fully implemented in compressed-domain. Firstly, summarization of generic videos is modeled as the process of extracted human objects under various activities/events. Accordingly, frames are classified into five categories via fuzzy decision including shot changes (cut and gradual transitions), motion activities (camera motion and object motion) and others by using two inter-frame measurements. Secondly, human objects are detected using Haar-like features. With the detected human objects and attained frame categories, activity levels for each frame are determined to adapt with video contents. Continuous frames belonging to same category are grouped to form one activity entry as content of interest (COI) which will convert the original video into a series of activities. An overall adjustable quota is used to control the size of generated summarization for efficient streaming purpose. Upon this quota, the frames selected for summarization are determined by evenly sampling the accumulated activity levels for content adaptation. Quantitative evaluations have proved the effectiveness and efficiency of our proposed approach, which provides a more flexible and general solution for this topic as domain-specific tasks such as accurate recognition of objects can be avoided. [Copyright &y& Elsevier]

Abstract Video summarization has unprecedented importance to help us overview current ever-growing amount of video collections. In this paper, we propose a novel dynamic video summarization model based on deep learning architecture. We are the first to solve the imbalanced class distribution problem in video summarization. The over-sampling algorithm is used to balance the class distribution on training data. The novel two-stream deep architecture with the cost-sensitive learning is proposed to handle the class imbalance problem in feature learning. In the spatial stream, RGB images are used to represent the appearance of video frames, and in the temporal stream, multi-frame motion vectors with deep learning framework is firstly introduced to represent and extract temporal information of the input video. The proposed method is evaluated on two standard video summarization datasets and a standard emotional dataset. Empirical validations for video summarization demonstrate that our model achieves performance improvement over the existing and state-of-the-art methods. Moreover, the proposed method is able to highlight the video content with the active level of arousal in affective computing task. In addition, the proposed frame-based model has another advantage. It can automatically preserve the connection between consecutive frames. Although the summary is constructed based on the frame level, the final summary is comprised of informative and continuous segments instead of individual separate frames. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new algorithm for fast estimation of camera motion directly in MPEG compressed domain, which starts from an existing and commonly used camera motion model to characterize the relationship between corresponding points of the neighboring frames, and then exploits the MPEG motion estimation and compensation techniques to derive closed-form estimation of all the three camera motion parameters, including pan, tilt, and zoom. The proposed algorithm features in extremely low computing cost and high processing speed with 50% improvement over the representative existing algorithms. It is suitable for implementation on hardware, such as Pan-Tilt-Zoom (PTZ) video camera. Furthermore, the three separated processing units (pan, tilt, and zoom) can be integrated into one simplified Fast PTZ Processing Unit. Comparative experiments also reveal that the proposed algorithm achieves similar estimation precision for all the three camera motion parameters. [ABSTRACT FROM AUTHOR]

• Summarize the law of eye fixations via statistical analysis on video saliency dataset. • Dominant salient object re-augmentation strategy for accurate saliency prediction. • Top-down semantic guided knowledge for salient object ranking. • Bottom-up feature weighting to re-shuffle the spatial pixel weights. Image saliency is determined by spatial semantic features, while video saliency is affected by multiple factors such as spatial and temporal information. Since human eyes stay extremely short on each frame, the dynamic salient area is more focused and concentrated on one salient object. In order to better simulate the human visual attention mechanism in dynamic scenes, we propose a key salient object re-augmentation method (KSORA) based on the guidance of both bottom-up weighted features and top-down semantic knowledge. The bottom-up feature weighting strategy effectively eliminates noisy and redundancy, and provide accurate local spatiotemporal features for saliency inference. The top-down key object enhancement strategy ranks salient candidates based on global statistical knowledge, so as to explicitly enhance the saliency proportion of the key object. The fusion of the local weighted spatiotemporal features and the global key object augmentation features not only ensures spatiotemporal consistency, but also facilitates obtaining more concentrated salient prediction. Results on three large datasets validate that our proposed method has the capability of improving the detection accuracy in complex scenes. [ABSTRACT FROM AUTHOR]