Back to Search
Start Over
Multiframe interpolation for video using phase features.
- Source :
-
Journal of Electronic Imaging . Jul/Aug2019, Vol. 28 Issue 4, p1-14. 14p. - Publication Year :
- 2019
-
Abstract
- Traditional frame interpolation algorithms typically find dense correspondences to synthesize an in-between frame. Finding correspondences is often sensitive to occlusion, disocclusion, and changes in color or luminance. We present a phase-feature-aided multiframe interpolation network that aims to estimate multiple in-between frames in one pass and handle challenging scenarios such as extreme light changes and occlusion. We first model the relation between multiple in-between frames together to enhance the temporal consistency. Two candidate optical flow fields are produced for a given in-between frame, one predicted from our network and the other estimated from those of neighboring frames using a flow fusion map. We also employ an image fusion map to combat occlusion problems in the warping processes, producing two candidate interpolated images that are fed to a shallow network with a residual structure to obtain the final interpolated image. To handle challenging scenarios, we apply a set of Gabor filters to extract phase variations in the feature domain with a multiscale phase subnetwork. Our entire neural network is end-to-end trainable. Our experiments show that this method outperforms the state-of-the-art approaches and achieves marked visual improvement in challenging scenarios. [ABSTRACT FROM AUTHOR]
- Subjects :
- *INTERPOLATION
*INTERPOLATION algorithms
*OPTICAL flow
*GABOR filters
*IMAGE fusion
Subjects
Details
- Language :
- English
- ISSN :
- 10179909
- Volume :
- 28
- Issue :
- 4
- Database :
- Academic Search Index
- Journal :
- Journal of Electronic Imaging
- Publication Type :
- Academic Journal
- Accession number :
- 138683882
- Full Text :
- https://doi.org/10.1117/1.JEI.28.4.043002