Back to Search
Start Over
Mixpred: Video Prediction Beyond Optical Flow
- Source :
- IEEE Access, Vol 7, Pp 185654-185665 (2019)
- Publication Year :
- 2019
- Publisher :
- IEEE, 2019.
-
Abstract
- Video prediction is a meaningful task for it has a wide range of application scenarios. And it is also a challenging task since it needs to learn the internal representation of a given video for both appearance and motion dynamics. The existing methods regard this problem as a spatiotemporal sequence forecasting problem and try to resolve it in a one-shot fashion, which causes the prediction result being blurry or inaccurate. So, a more intuitional thought is to split this problem into two parts: model the dynamic pattern of the given video and learn the appearance representation of given video frames. In this paper, we develop a novel network structure named MixPred based on this idea to address this issue. We divide the prediction problem into two parts as mentioned above and build two subnets to solve these two parts separately. Instead of fusing the results of subnets at the final layer, we put forward a parallel interaction style through the whole process to merge the dynamic information and content information in a more natural way. Besides, we propose three different connection methods for exploring the most effective connection structure. We trained the model on UCF-101 and KITTI, and testing our model on UCF-101, KITTI, and Caltech. The results demonstrate that our method achieves state-of-the-art both quantitatively and qualitatively.
- Subjects :
- General Computer Science
Computer science
business.industry
video prediction
spatial subnet
General Engineering
Optical flow
02 engineering and technology
Video prediction
010501 environmental sciences
01 natural sciences
parallel interaction
0202 electrical engineering, electronic engineering, information engineering
End-to-end training
020201 artificial intelligence & image processing
General Materials Science
Artificial intelligence
lcsh:Electrical engineering. Electronics. Nuclear engineering
business
lcsh:TK1-9971
0105 earth and related environmental sciences
temporal subnet
Subjects
Details
- Language :
- English
- ISSN :
- 21693536
- Volume :
- 7
- Database :
- OpenAIRE
- Journal :
- IEEE Access
- Accession number :
- edsair.doi.dedup.....ca653d663bb151d479929a85ab09087e