Search

360-degree videos, also known as omnidirectional or panoramic videos, provide the user an immersive experience that 2D videos cannot provide. It is crucial to access the perceived quality of the 360-degree video. 2D video quality assessment (VQA) methods are unsuitable for 360-degree videos. There are few 360-degree video quality assessment (360VQA) methods. This paper proposes a multi-viewport based 3D convolutional neural network for 360VQA (3D-360VQA). First, it is easy to divide the 2D planar video into rectangular blocks as video patches in order to adapt to a deep neural network. The way to form the video patch in a 2D planar video is unsuitable for a 360-degree video. Thus, a multiple viewports based video patch forming method is proposed. Second, although the deep neural networks have achieved great success in image quality assessment (IQA), there are few deep neural networks for 360VQA. A 3D convolution based deep neural network is proposed to predict the perceived quality of 360-degree videos. The publicly available 360-degree videos datasets are used to evaluate the proposed method. The experimental results show that the proposed method is suitable for the 360-degree video and outperforms other existing methods, which verifies the effectiveness of our network architecture. [ABSTRACT FROM AUTHOR]

Video coding removes spatial, temporal, and statistic redundancies. After H.265, to further improve the coding efficiency, many efforts have been dedicated to removing the perceptual redundancy by using human perception-based methods. Just noticeable difference (JND) gives a good approximation for the human visual system and provides a valuable solution to remove the perceptual redundancy for perceptual video coding (PVC). However, there are still problems in the PVC architecture and the JND profile. One is although the whole discrete cosine transform (DCT) block are suppressed, there are still many transform coefficients below the suppression levels which are not adequately suppressed. Another problem is, to the best of our knowledge, most JND profiles are measured by image-based test methods and past display equipment. However, compared to images, videos exhibits temporal characteristics, and the current trend of the display equipment is towards full high definition. To solve these problems, we first propose a high efficiency video coding (HEVC)-compliant PVC architecture, where the coefficients in a DCT block can be adequately suppressed in a whole block manner. Second, we propose a video-based test method to model the temporal masking (TM) effect, called TM-JND. Experimental results show that the proposed TM-JND model can more accurately estimate the JND values for today's display equipment and videos, avoiding the overestimate of the JND values like other existing models. The proposed PVC architecture achieves a significant bitrate reduction with a negligible subjective quality loss, compared with the HEVC test model HM 16.9. [ABSTRACT FROM AUTHOR]

Omnidirectional video quality assessment (OVQA) helps to evaluate the viewers' visual experience and promotes the development of omnidirectional video. The perceived quality of omnidirectional video is affected not only by the video content and distortion, but also by the viewing directions of viewer's preference. At present, there are some quality assessment methods for omnidirectional video, but most of them are full-reference (FR). Compared with the FR method, the no-reference (NR) method becomes more difficult due to the lack of the reference video. In this paper, a NR OVQA based on generative adversarial networks (GAN) is proposed, which is composed of the reference video generator and the quality score predictor. Generally, a reference image/video is distorted by some distortion types, and each distortion type has some distortion levels. To the best of our knowledge, there are some NR methods using GAN to generate the reference images/videos for quality assessment. In these methods, the reference images/videos is generated by GAN when the distorted images/videos, which are from a distortion type but with different distortion level, are input into the GAN. In order to achieve an accurate quality assessment, the generated reference images/videos, which are from a distortion type but with different distortion level, are expected to have as similar quality as possible with each other. However, the distorted images/videos are independent for GAN, and the GAN will generate a little different reference images/videos for these distorted images/videos. This issue is not considered in the existing GAN-based methods. To solve this issue, we introduced a level loss in OVQA. For the quality score predictor, as a further contribution of this paper, the viewing direction of the omnidirectional video is incorporated to guide the quality and weight regression. The publicly available dataset is used to evaluate the proposed method. The experimental results indicate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Omnidirectional video quality assessment (OVQA) helps to evaluate the viewers' visual experience and promotes the development of omnidirectional video. The perceived quality of omnidirectional video is affected not only by the video content and distortion, but also by the viewing directions of viewer's preference. At present, there are some quality assessment methods for omnidirectional video, but most of them are full-reference (FR). Compared with the FR method, the no-reference (NR) method becomes more difficult due to the lack of the reference video. In this paper, a NR OVQA based on generative adversarial networks (GAN) is proposed, which is composed of the reference video generator and the quality score predictor. Generally, a reference image/video is distorted by some distortion types, and each distortion type has some distortion levels. To the best of our knowledge, there are some NR methods using GAN to generate the reference images/videos for quality assessment. In these methods, the reference images/videos is generated by GAN when the distorted images/videos, which are from a distortion type but with different distortion level, are input into the GAN. In order to achieve an accurate quality assessment, the generated reference images/videos, which are from a distortion type but with different distortion level, are expected to have as similar quality as possible with each other. However, the distorted images/videos are independent for GAN, and the GAN will generate a little different reference images/videos for these distorted images/videos. This issue is not considered in the existing GAN-based methods. To solve this issue, we introduced a level loss in OVQA. For the quality score predictor, as a further contribution of this paper, the viewing direction of the omnidirectional video is incorporated to guide the quality and weight regression. The publicly available dataset is used to evaluate the proposed method. The experimental results indicate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]