1. Video Super-Resolution Reconstruction Based on Deep Learning and Spatio-Temporal Feature Self-Similarity
- Author
-
Junping Du, Feifei Kou, Zhe Xue, Linghui Li, Meiyu Liang, Xiaoxiao Wang, and Wang Xu
- Subjects
Self-similarity ,business.industry ,Computer science ,Deep learning ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Pattern recognition ,Similarity measure ,Mixture model ,Convolutional neural network ,Computer Science Applications ,Computational Theory and Mathematics ,Similarity (network science) ,Feature (computer vision) ,Artificial intelligence ,Noise (video) ,business ,Information Systems - Abstract
To address the problems in the existing video super-resolution methods, such as noise, over smooth and visual artifacts, which are caused by reliance on limited external training or mismatch of internal similarity instances, this study proposes a video super-resolution reconstruction algorithm based on deep learning and spatio-temporal feature similarity (DLSS-VSR). The video super-resolution reconstruction mechanism with joint internal and external constraints is established utilizing both external deep correlation mapping learning and internal spatio-temporal nonlocal self-similarity prior constraint. A deep learning model based on deep convolutional neural network is constructed to learn the nonlinear correlation mapping between low-resolution and high-resolution video frame patches. A spatio-temporal feature similarity calculation method is proposed, which considers both internal video spatio-temporal self-similarity and external clean nonlocal similarity. For the internal spatio-temporal feature self-similarity, we improve the accuracy and robustness of similarity matching by proposing a similarity measure strategy based on spatio-temporal moment feature similarity and structural similarity. The external nonlocal similarity prior constraint is learned by patch group-based Gaussian mixture model. The time efficiency for spatio-temporal similarity matching is further improved based on saliency detection and region correlation judgment strategy. Experimental results demonstrate that the DLSS-VSR achieves competitive super-resolution quality compared to other state-of-the-art algorithms.
- Published
- 2022