Search

Blind image quality assessment (BIQA) has witnessed a flourishing progress due to the rapid advances in deep learning technique. The vast majority of prior BIQA methods try to leverage models pre-trained on ImageNet to mitigate the data shortage problem. These well-trained models, however, can be sub-optimal when applied to BIQA task that varies considerably from the image classification domain. To address this issue, we make the first attempt to leverage the plentiful unlabeled data to conduct self-supervised pre-training for BIQA task. Based on the distorted images generated from the high-quality samples using the designed distortion augmentation strategy, the proposed pre-training is implemented by a feature representation prediction task. Specifically, patch-wise feature representations corresponding to a certain grid are integrated to make prediction for the representation of the patch below it. The prediction quality is then evaluated using a contrastive loss to capture quality-aware information for BIQA task. Experimental results conducted on KADID-10 k and KonIQ-10 k databases demonstrate that the learned pre-trained model can significantly benefit the existing learning based IQA models. [ABSTRACT FROM AUTHOR]

With the increasing prevalence of free-viewpoint video applications, virtual view synthesis has attracted extensive attention. In view synthesis, a new viewpoint is generated from the input color and depth images with a depth-image-based rendering (DIBR) algorithm. Current quality evaluation models for view synthesis typically operate on the synthesized images, i.e. after the DIBR process, which is computationally expensive. So a natural question is that can we infer the quality of DIBR-based synthesized images using the input color and depth images directly without performing the intricate DIBR operation. With this motivation, this paper presents a no-reference image quality prediction model for view synthesis via COlor-Depth Image Fusion, dubbed CODIF, where the actual DIBR is not needed. First, object boundary regions are detected from the color image, and a Wavelet-based image fusion method is proposed to imitate the interaction between color and depth images during the DIBR process. Then statistical features of the interactional regions and natural regions are extracted from the fused color-depth image to portray the influences of distortions in color/depth images on the quality of synthesized views. Finally, all statistical features are utilized to learn the quality prediction model for view synthesis. Extensive experiments on public view synthesis databases demonstrate the advantages of the proposed metric in predicting the quality of view synthesis, and it even suppresses the state-of-the-art post-DIBR view synthesis quality metrics. [ABSTRACT FROM AUTHOR]

Though blind image quality assessment (BIQA) is highly desired in perceptual-oriented image processing systems, it is extremely difficult to design a reliable BIQA method. With the help of the prior knowledge, the human visual system (HVS) hierarchically perceives the quality degradation during the visual recognition. Inspired by this, we suggest different levels of distortion generate individual degradations on hierarchical features, and propose to consider the degradations on both low and high level features for quality prediction. By mimicking the orientation selectivity (OS) mechanism in the primary visual cortex, an OS based local structure is designed for low-level visual information representation. At the meantime, the deep residual network, which possesses multiple levels for feature integration, is employed to extract the deep semantics for high-level visual content representation. By fusing the local structure and the deep semantics, a hierarchical feature set is acquired. Next, the correlations between the degradations of image qualities and their corresponding hierarchical feature sets are analyzed, and a novel hierarchical feature degradation (HFD) based BIQA (HFD-BIQA) method is built. Experimental results on the legacy and wild image quality assessment databases demonstrate the prediction accuracy of the proposed HFD-BIQA method, and verify that the HFD-BIQA performs highly consistent with the subjective perception. [ABSTRACT FROM AUTHOR]

Highlights • A novel blind image quality assessment is proposed. • The proposed method evaluates distorted image from a completely new perspective of human subjective perception. • The experiment results demonstrate the superiority of the proposed method. Abstract No-reference (NR) image quality assessment (IQA) aims to evaluate the quality of an image without reference image, which is greatly desired in the automatic visual signal processing system. Distortions degrade the visual contents and affect the semantics acquisition during the process of human perception. Although the existing methods evaluate the quality of images based on the structure, texture, or statistical characteristics, and deliver high quality prediction accuracy, they do not take the spatial semantics into account. From the perspective of human perception, distortions decrease the structural semantics that represent the structural information, and disturb the spatial semantics that describe the contents of images. Therefore, we attempt to measure the image quality by its degradation of semantics in an image. To extract the semantics of an image, a semantic network is proposed. The network contains convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) that correspond to structural semantics and spatial semantics, respectively. CNN can be regarded as a coarse imitation of human visual mechanism to obtain the structural information, and LSTM can express the contents of an image. Then, by measuring the degradations of different semantics on images, a novel NR IQA is introduced. The proposed approach is evaluated on the databases of LIVE, CSIQ, TID2013, and LIVE multiply distorted database as well as LIVE in the wild image quality challenge database, and the results show superior performance to other state-of-the-art NR IQA methods. Furthermore, we explore the generalization capability of the proposed approach, and the experimental results indicate the proposed approach has a high robustness. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel no reference quality assessment method by incorporating statistical luminance and texture features (NRLT) for screen content images (SCIs) with both local and global feature representation. The proposed method is designed inspired by the perceptual property of the human visual system (HVS) that the HVS is sensitive to luminance change and texture information for image perception. In the proposed method, we first calculate the luminance map through the local normalization, which is further used to extract the statistical luminance features in global scope. Second, inspired by existing studies from neuroscience that high-order derivatives can capture image texture, we adopt four filters with different directions to compute gradient maps from the luminance map. These gradient maps are then used to extract the second-order derivatives by local binary pattern. We further extract the texture feature by the histogram of high-order derivatives in global scope. Finally, support vector regression is applied to train the mapping function from quality-aware features to subjective ratings. Experimental results on the public large-scale SCI database show that the proposed NRLT can achieve better performance in predicting the visual quality of SCIs than relevant existing methods, even including some full reference visual quality assessment methods. [ABSTRACT FROM PUBLISHER]

Reduced-reference (RR) image quality assessment (IQA) aims to use less reference data for quality evaluation. Global features are demanded to effectively express quality degradation caused by distortion. Since the human visual system (HVS) is highly sensitive to structure degradation, we suggest to represent the visual content of an image with several structural patterns. Also, image quality is measured based on the structural degradation on these patterns. Firstly, the spatial correlation of image structure is analyzed with the local binary pattern (LBP), and several representative structural patterns are extracted. Then, the energy changes on these patterns between the reference image and the distorted image are calculated. Finally, the support vector regression (SVR) procedure is adopted for feature pooling, and the quality score of the input image is returned. Experimental results for three publicly available databases demonstrate that the proposed RR IQA method is highly consistent with the human perception, which uses a small amount of reference data (20 values) and achieves a promising improvement on quality evaluation accuracy. [ABSTRACT FROM AUTHOR]

Objective image quality assessment (IQA) aims to evaluate image quality consistently with human perception. Most of the existing perceptual IQA metrics cannot accurately represent the degradations from different types of distortion, e.g., existing structural similarity metrics perform well on content-dependent distortions while not as well as peak signal-to-noise ratio (PSNR) on content-independent distortions. In this paper, we integrate the merits of the existing IQA metrics with the guide of the recently revealed internal generative mechanism (IGM). The IGM indicates that the human visual system actively predicts sensory information and tries to avoid residual uncertainty for image perception and understanding. Inspired by the IGM theory, we adopt an autoregressive prediction algorithm to decompose an input scene into two portions, the predicted portion with the predicted visual content and the disorderly portion with the residual content. Distortions on the predicted portion degrade the primary visual information, and structural similarity procedures are employed to measure its degradation; distortions on the disorderly portion mainly change the uncertain information and the PNSR is employed for it. Finally, according to the noise energy deployment on the two portions, we combine the two evaluation results to acquire the overall quality score. Experimental results on six publicly available databases demonstrate that the proposed metric is comparable with the state-of-the-art quality metrics. [ABSTRACT FROM PUBLISHER]

No-reference/blind image quality assessment (BIQA) is designed to measure the image quality without any knowledge of the reference image. Most existing BIQA metrics employ natural scene statistics or learning based models, which have achieved great progress but there still remains much room for improvement. Cognitive researches indicate that natural images possess sparse structures which can be represented by a small number of descriptors. Considering the sparse property of images, we utilize the bag-of-words (BoW) model for image representation and propose a novel BIQA metric. By analyzing the effect of neighboring pixel number and quantization depth of local pattern on image content extraction, we adopt the local quantized pattern (LQP) to extract image feature descriptors. The codebook is constructed by clustering LQP based descriptors from a set of natural images instead of distorted images, which presents strong generalization ability. The BoW-based image feature representation is highly sensitive to various distortion types and levels. Experiments on three public databases verify the effectiveness of the proposed metric and indicate our method is highly consistent with human perception. [ABSTRACT FROM AUTHOR]

Compressive sensing (CS) has been attracting tremendous attention in recent years. Extensive CS image recovery algorithms have been proposed for more effective image reconstruction. However, very little work has been addressed for the quality assessment of CS recovered images, which may hinder further development of CS image recovery techniques. The CS recovered images are typically contaminated by multiple distortions, particularly at low sampling rates. Unfortunately, the existing quality metrics are ineffective for evaluating multiple distortions, so they are limited in predicting the quality of CS recovered images. Motivated by this, this paper presents a no-reference CS Recovered Image Quality (CSRIQ) metric based on the measurement of both local and global distortions in CS recovered images. The local features consist of a local phase coherence based edge sharpness measure and a gray level co-occurrence matrix based texture measure. The global features are extracted based on the natural scene statistics and can be divided into two types. One type is obtained by calculating image naturalness parameters. The other type is computed based on the statistics of singular value decomposition coefficients. Support vector regression is employed to do model training and the subsequent quality prediction. Experimental results conducted on a CS recovered image database demonstrate the advantages of the proposed method. As a application, the proposed metric is used for automatic parameter selection for CS image recovery algorithms. [ABSTRACT FROM AUTHOR]

Images are subject to blocking artifacts when they are compressed using the JPEG standard. Knowing the extent of blocking artifacts is thus necessary for such applications as automatic quality monitoring and restoration. The current blocking artifacts measures are based on a strong prior that the block boundaries are known in advance, which often does not hold in real-world applications. Therefore, their performances degrade significantly when block boundaries are misaligned. To address the problem, this paper presents a robust no-reference blocking artifacts evaluation metric for JPEG images based on grid strength and regularity (GridSAR). The underlying idea is to extract block grids from a JPEG image and quantify the grid image to evaluate the strength of blocking artifacts. To this end, a grid map of blocking artifacts is first extracted from the image in the spatial domain. Then the blocking artifacts of the image are evaluated by quantifying the strength and regularity of the grid image. Furthermore, in order to account for the varying sensitivities of human eyes to the blocking artifacts in smooth and textured areas, a masking function is also proposed. Experimental results on seven popular image quality databases demonstrate that GridSAR achieves the state-of-the-art performance, and is robust to block misalignments. [ABSTRACT FROM AUTHOR]

Image restoration is the process of restoring latent clear images from degraded images, and it has received substantial attention in various restoration scenarios recently. Contrary to the considerable progress of image restoration algorithms, the quality evaluation for image restoration falls behind, which may hinder the further development of advanced image restoration techniques. So far, only several objective quality metrics have been proposed to evaluate the quality of restored images for specific restoration scenarios, and little work has been dedicated to the applications of image restoration quality metrics. Besides, the performance of these metrics remains an open problem, and an obvious disadvantage of these metrics is that their robustness is quite weak. To bridge this gap, we present this survey paper. First, the difference between the traditional image quality assessment and quality assessment for image restoration was analyzed profoundly. Then, for different image restoration scenarios, this paper provides (i) a comprehensive description of the existing subjective quality databases, (ii) a thorough insight into existing objective quality metrics and (iii) two applications of image restoration quality metrics, namely, parameter selection and benchmarking image restoration algorithms. After that, the experimental results and a detailed performance analysis on (i) quality assessment of restored images, (ii) benchmarking image restoration algorithms and (iii) time complexity are given based on public restored image databases. Finally, the paper outlines the challenges and future trends of subjective and objective quality assessment for image restoration in various aspects. • The difference between the traditional image quality assessment and quality assessment for image restoration was analyzed profoundly. • For different image restoration scenarios, this paper provides a comprehensive description of the existing i) subjective quality databases, ii) objective quality metrics and iii) two applications of image restoration quality metrics. • A detailed performance analysis on i) quality assessment of restored images, ii) benchmarking image restoration algorithms and iii) time complexity are given based on public restored image databases. • The paper outlines the challenges and future trends of subjective and objective quality assessment for image restoration in various aspects. [ABSTRACT FROM AUTHOR]

Image quality assessment (IQA) is a fundamental problem in image processing. While in practice almost all images are represented in the color format, most of the current IQA metrics are designed in gray-scale domain. Color influences the perception of image quality, especially in the case where images are subject to color distortions. With this consideration, this paper presents a novel color image quality index based on Sparse Representation and Reconstruction Residual (SRRR). An overcomplete color dictionary is first trained using natural color images. Then both reference and distorted images are represented using the color dictionary, based on which two feature maps are constructed to measure structure and color distortions in a holistic manner. With the consideration that the feature maps are insensitive to image contrast change, the reconstruction residuals are computed and used as a complementary feature. Additionally, luminance similarity is also incorporated to produce the overall quality score for color images. Experiments on public databases demonstrate that the proposed method achieves promising performance in evaluating traditional distortions, and it outperforms the existing metrics when used for quality evaluation of color-distorted images. [ABSTRACT FROM AUTHOR]