Search

Blind image quality assessment (BIQA) is a fundamental yet challenging problem in the image processing system. Existing BIQA models have the following problems: 1) Due to the lack of available quality-label images, most of the methods have poor generalization ability in different distortion categories; 2) The impact of human visual characteristics on the content of images is not taken into account. In this paper, we proposed a visual residual perception optimized network (VRPON) that can effectively solve these problems. The proposed method separates the training of BIQA into two stages: 1) a distortion degree identification network and 2) an image quality prediction network. In the first stage, the spatial and temporal features of image sequences are extracted by CNN and LSTM respectively, which are used to evaluate the degree of image distortion. And then the proposed model is learned to predict image patches' scores in the second stage with the outputs of the first stage. Finally, a pooling strategy that follows the human visual saliency is designed to evaluate the quality score of the whole image. Experimental results show that the proposed VRPON not only has better performance than state-of-the-art methods on synthetic distorted images (LIVE, TID2013, CSIQ), but also has better robustness for different authentic distortions (LIVE challenge).

Video-based person re-identification (Re-ID) remains to be a promising but challenging computer vision task, suffering from a lack of discriminative features that better aggregate both the spatial and temporal information. In this paper, we propose a joint attentive spatial-temporal feature aggregation network (JAFN) for the video-based person Re-ID, simultaneously learning the quality- and frame-aware model to obtain attention-based spatial-temporal feature aggregation. Specifically, we utilize CNN to learn the spatial features, while introducing the LSTM to separately learn the temporal features. For the feature aggregation, we introduce two attention mechanisms respectively for generating the quality and frame significance score, where the quality score measures the quality of the images for attentive spatial feature aggregation, and the frame score measures the significance of the image frames contributing to the temporal feature. Then, we utilize the set-pooling for both the quality-aware spatial feature and the frame-aware temporal feature aggregation based on the attentive scores. The residual learning is also introduced to play between the LSTM and the CNN for adaptive spatial-temporal feature fusion. Furthermore, we adopt the data balance to alleviate the data disproportions existing in datasets of the video-based Re-ID. The extensive experimental results conducted on the PRID2011, i-LIDS-VID, and MARS datasets demonstrate the effectiveness of the proposed JAFN. Furthermore, comparison results conducted on different modules and features in the JAFN show that our approach is of favorable generalization ability on attentively aggregating both the spatial and temporal features.

2D image quality assessment (IQA) and stereoscopic 3D IQA are considered as two different tasks in the literature. In this paper, we present an index for both no-reference 2D and 3D IQA. We propose to transform the IQA task into a task of quality comparison between images. By generating image pairs, the amount of training data reaches the square of the original amount of data, effectively solving the lacking of training samples. We also propose a learning to rank model using Siamese convolutional neural networks (LRSN) for quality comparison. The presented LRSN has two branches that have the same structure, share weights with each other, and take two image patches as inputs. The goal of LRSN is learning to rank the quality scores between the two input image patches. The relative quality score of a test image is obtained by first comparing its image patches with many image patches of other images and counts the number of times that its image patches are ranked superior to other patches. The experimental results on three 2D (LIVE, CSIQ, and TID2013) and three 3D (LIVE 3D Phase-I, LIVE 3D Phase-II, and NBU) IQA databases demonstrate that the proposed LRSN model works well for both 2D and 3D IQA and outperforms the state-of-the-art no-reference 2D and 3D IQA metrics.