Back to Search Start Over

Real-Time Video Saliency Prediction Via 3D Residual Convolutional Neural Network

Authors :
Zhenhao Sun
Xu Wang
Qiudan Zhang
Jianmin Jiang
Source :
IEEE Access, Vol 7, Pp 147743-147754 (2019)
Publication Year :
2019
Publisher :
IEEE, 2019.

Abstract

Attention is a fundamental attribute of human visual system that plays important roles in many visual perception tasks. The key issue of video saliency lies in how to efficiently exploit the temporal information. Instead of singling out the temporal saliency maps, we propose a real-time end-to-end video saliency prediction model via 3D residual convolutional neural network (3D-ResNet), which incorporates the prediction of spatial and temporal saliency maps into one single process. In particular, a multi-scale feature representation scheme is employed to further boost the model performance. Besides, a frame skipping strategy is proposed for speeding up the saliency map inference process. Moreover, a new challenging eye tracking database with 220 video clips is established to facilitate the research of video saliency prediction. Extensive experimental results show our model outperforms the state-of-the-art methods over the eye fixation datasets in terms of both prediction accuracy and inference speed.

Details

Language :
English
ISSN :
21693536
Volume :
7
Database :
Directory of Open Access Journals
Journal :
IEEE Access
Publication Type :
Academic Journal
Accession number :
edsdoj.0bb9f39c5e424c80a9e1cbcdbaf99b77
Document Type :
article
Full Text :
https://doi.org/10.1109/ACCESS.2019.2946479