Search

Video quality assessment (VQA) remains an important and challenging problem that affects many applications at the widest scales. Recent advances in mobile devices and cloud computing techniques have made it possible to capture, process, and share high resolution, high frame rate (HFR) videos across the Internet nearly instantaneously. Being able to monitor and control the quality of these streamed videos can enable the delivery of more enjoyable content and perceptually optimized rate control. Accordingly, there is a pressing need to develop VQA models that can be deployed at enormous scales. While some recent effects have been applied to full-reference (FR) analysis of variable frame rate and HFR video quality, the development of no-reference (NR) VQA algorithms targeting frame rate variations has been little studied. Here, we propose a first-of-a-kind blind VQA model for evaluating HFR videos, which we dub the Framerate-Aware Video Evaluator w/o Reference (FAVER). FAVER uses extended models of spatial natural scene statistics that encompass space–time wavelet-decomposed video signals, and leverages the advantages of the deep neural network to provide motion perception, to conduct efficient frame rate sensitive quality prediction. Our extensive experiments on several HFR video quality datasets show that FAVER outperforms other blind VQA algorithms at a reasonable computational cost. To facilitate reproducible research and public evaluation, an implementation of FAVER is being made freely available online: https://github.com/uniqzheng/HFR-BVQA. [ABSTRACT FROM AUTHOR]

Abstract Blind/no-reference image quality assessment (NR-IQA) aims to assess the quality of an image without any reference image. In this paper, we propose an effective and efficient general-purpose NR-IQA model using natural scene statistics (NSS) of the luminance relative order, based on the observation that the variation of the marginal distribution of the relative order coefficients effectively reflect the degree of warping caused by different types of image distortions. In the literature, gradient-relevant methods have had a big success in full-reference (FR) IQA and reduced-reference (RR) IQA. Inspired by these, we extend it to NR-IQA in this paper. Notice that the NSS-based models usually extract their features derived from the spatial, wavelet, DCT and spectral domain etc. Unlike these metrics, the proposed method firstly extracts 32 natural scene statistics features of the luminance relative order, obtained from the log histograms of log horizontal, vertical, main-diagonal and secondary-diagonal derivatives, along with kurtosis, variance, differential entropy and entropy at two scales. Then a mapping is learned to predict the quality score using a support vector regression. The experimental results on several benchmark databases showed that the proposed method is comparable with the state-of-the-art methods and has a relatively low complexity. [ABSTRACT FROM AUTHOR]

Human visual system (HVS) identifies each unit of stereo-pair into binocular or monocular perception depending on distortions and depth perception. However, these HVS properties still have a large room for exploration in stereoscopic image quality assessment (SIQA) research field. In this paper, a bivariate natural scene statistics (NSS) model is proposed to capture image quality by extracting features from binocular and monocular perception regions, respectively. In the implementation details, the stereo-pair is first segmented into various regions based on spatial information of its disparity. Then the regions are classified into categories of binocular fusion, binocular rivalry and binocular suppression. Bivariate statistics of spatially adjacent Gabor response of image are extracted from each category of regions, based on which features are calculated for image quality representation. In particular, the extraction strategy depends on the type of image patch. Experimental results show that the proposed model is promising at handling the task of SIQA on LIVE 3D Image Quality Database. [ABSTRACT FROM AUTHOR]

Measuring visual quality, as perceived by human observers, is becoming increasingly important in a large number of applications where humans are the ultimate consumers of visual information. Many natural image databases have been developed that contain human subjective ratings of the images. Subjective quality evaluation data is less available for synthetic images, such as those commonly encountered in graphics novels, online games or internet ads. A wide variety of powerful full-reference, reduced-reference and no-reference Image Quality Assessment (IQA) algorithms have been proposed for natural images, but their performance has not been evaluated on synthetic images. In this paper we (1) conduct a series of subjective tests on a new publicly available Embedded Signal Processing Laboratory (ESPL) Synthetic Image Database, which contains 500 distorted images (20 distorted images for each of the 25 original images) in 1920 × 1080 resolution, and (2) evaluate the performance of more than 50 publicly available IQA algorithms on the new database. The synthetic images in the database were processed by post acquisition distortions, including those arising from compression and transmission. We collected 26,000 individual ratings from 64 human subjects which can be used to evaluate full-reference, reduced-reference, and no-reference IQA algorithm performance. We find that IQA models based on scene statistics models can successfully predict the perceptual quality of synthetic scenes. The database is available at: http://signal.ece.utexas.edu/%7Ebevans/synthetic/ . [ABSTRACT FROM AUTHOR]

Compressive sensing (CS) has been attracting tremendous attention in recent years. Extensive CS image recovery algorithms have been proposed for more effective image reconstruction. However, very little work has been addressed for the quality assessment of CS recovered images, which may hinder further development of CS image recovery techniques. The CS recovered images are typically contaminated by multiple distortions, particularly at low sampling rates. Unfortunately, the existing quality metrics are ineffective for evaluating multiple distortions, so they are limited in predicting the quality of CS recovered images. Motivated by this, this paper presents a no-reference CS Recovered Image Quality (CSRIQ) metric based on the measurement of both local and global distortions in CS recovered images. The local features consist of a local phase coherence based edge sharpness measure and a gray level co-occurrence matrix based texture measure. The global features are extracted based on the natural scene statistics and can be divided into two types. One type is obtained by calculating image naturalness parameters. The other type is computed based on the statistics of singular value decomposition coefficients. Support vector regression is employed to do model training and the subsequent quality prediction. Experimental results conducted on a CS recovered image database demonstrate the advantages of the proposed method. As a application, the proposed metric is used for automatic parameter selection for CS image recovery algorithms. [ABSTRACT FROM AUTHOR]

Assessing the quality of images is a challenging task. To achieve this goal, images must be evaluated by a pool of subjects following a well-defined protocol or an objective quality metric must be defined. In this work, an objective quality metric based on deep neural network is proposed. The metric takes into account the human vision system by computing the saliency map and natural scene statistics features of the image under test. The neural network is composed by two modules: the convolutional layers and the regression units. The first one is trained by using preprocessed distorted images. The feature weights of the first module are smoothed by exploiting the estimated saliency map. The latter module is fit with the ground truth quality scores of the input image and the scaled feature weights obtained from first module by using visual sensitivity factor of image obtained using natural scene statistics features. The performances of the proposed metric have been evaluated by using four datasets: LIVEIQA, TID2013, CSIQ, and KADID10K. The achieved results show the effectiveness of the proposed system in closely matching the predicted quality scores with the ground truth ones. • It is a no-reference approach based on a fully deep neural-network • The saliency estimation model extracts the visually most relevant area of the image. • Using visual sensitivity for scaling the feature weights improves the performance. [ABSTRACT FROM AUTHOR]

We present two contributions in this work: (i) a bivariate generalized Gaussian distribution (BGGD) model for the joint distribution of luminance and disparity subband coefficients of natural stereoscopic scenes and (ii) a no-reference (NR) stereo image quality assessment algorithm based on the BGGD model. We first empirically show that a BGGD accurately models the joint distribution of luminance and disparity subband coefficients. We then show that the model parameters form good discriminatory features for NR quality assessment. Additionally, we rely on the previously established result that luminance and disparity subband coefficients of natural stereo scenes are correlated, and show that correlation also forms a good feature for NR quality assessment. These features are computed for both the left and right luminance-disparity pairs in the stereo image and consolidated into one feature vector per stereo pair. This feature set and the stereo pair׳s difference mean opinion score (DMOS) (labels) are used for supervised learning with a support vector machine (SVM). Support vector regression is used to estimate the perceptual quality of a test stereo image pair. The performance of the algorithm is evaluated over popular databases and shown to be competitive with the state-of-the-art no-reference quality assessment algorithms. Further, the strength of the proposed algorithm is demonstrated by its consistently good performance over both symmetric and asymmetric distortion types. Our algorithm is called Stereo QUality Evaluator (StereoQUE). [ABSTRACT FROM AUTHOR]

In this paper, we introduce a completely blind and unsupervised no-reference model that performs the quality assessment of stereoscopic 3D videos by using joint motion and depth visibility. In order to achieve such objective metric, first we compute the correlation map between the motion vector map and the depth map, and then we study the natural scene statistics of the estimated correlation map. We empirically model these statistics with Univariate Generalized Gaussian Distribution (UGGD) and compute the UGGD parameters at multi-scale and multi-orient steerable subband decomposition. It is shown that the estimated UGGD features are capable of discriminating distortions. We then compute the mean vector and the covariance matrix from the estimated UGGD features of the correlation maps of motion vector and the depth maps of undistorted and distorted S3D videos. This is followed by the computation of the Wave Hedges distance between the mean vectors of the undistorted and distorted contents, and the Bhattacharyya distance is measured between the corresponding covariance matrices. We pool these distance measures to achieve the overall joint motion and depth quality of S3D videos. The efficiency of the proposed algorithm is evaluated on the well-known IRCCYN, LFOVIAS3DPh1 and LFOVIAS3DPh2 S3D video datasets. Our model demonstrates robust and consistent performance across all distortion types and shows competitive performance against other 2D and 3D image and video quality assessment algorithms. Furthermore, since the proposed algorithm is completely blind, it does not require any training and testing analysis on the content features and the subjective scores. [ABSTRACT FROM AUTHOR]

We study the problem of joint low light image contrast enhancement and denoising using a statistical approach. The low light natural image in the band pass domain is modeled by statistically relating a Gaussian scale mixture model for the pristine image, to the low light image, through a detail loss coefficient and Gaussian noise. The detail loss coefficient is statistically described using a posterior distribution with respect to its estimate based on a prior contrast enhancement algorithm. We then design our low light enhancement and denoising (LLEAD) method by computing the minimum mean squared error estimate of the pristine image band pass coefficients. We create the Indian Institute of Science low light image dataset of well-lit and low light image pairs to learn the model parameters and evaluate our enhancement method. We show through extensive experiments on multiple datasets that our method helps better enhance the contrast while simultaneously controlling the noise when compared to other state of the art joint contrast enhancement and denoising methods. [ABSTRACT FROM AUTHOR]

No-reference image quality assessment (NR-IQA) aims to develop models that can predict the quality of distorted image automatically and accurately in the absent of reference image. Previous NR-IQA methods based on natural scene statistics (NSS) always focus on the luminance contrast of image but attach limited attention to pixel-wise relationship. However, human visual system (HVS) is highly adaptive to extract spatial correlation according to relative position within visual field. In this paper, a new approach is proposed for NR-IQA, in which the neighborhood co-occurrence matrix (NCM) is introduced to describe spatial correlation of pixels for quality assessment. The NCM is constructed based on spatial correlation of every pixel and its neighborhood through a mapping to highlight the one-to-many pixel-wise relationship. Moreover, a series of tailored statistical metrics are designed to quantify the unnaturalness extent of NCM effectively, which is combined with others natural scene statistics to predict image quality. Extensive experiments demonstrate the proposed method has superior performance against compared methods, and achieves significant improvements on distortions associated with color or locality. • The significant of spatial correlation of pixels for quality evaluation is analyzed. • The neighborhood co-occurrence matrix and three metrics are introduced for NR-IQA. • The new approach performs better on distortions associated with color or locality. [ABSTRACT FROM AUTHOR]