Search

Underwater imaging is often affected by light attenuation and scattering in water, leading to degraded visual quality, such as color distortion, reduced contrast, and noise. Existing underwater image enhancement (UIE) methods often lack generalization capabilities, making them unable to adapt to various underwater images captured in different aquatic environments and lighting conditions. To address these challenges, a UIE method based on the conditional denoising diffusion probabilistic model (DDPM) is proposed (DiffWater), which leverages the advantages of DDPM and trains a stable and well-converged model capable of generating high-quality and diverse samples. Considering the multiple distortion issues in underwater imaging, unconditional DDPM may not achieve satisfactory enhancement and restoration results. Therefore, DiffWater utilizes the degraded underwater image with added color compensation as a conditional guide, through which the DiffWater achieves high-quality restoration of degraded underwater images. Particularly, the proposed DiffWater introduces a color compensation method that performs channelwise color compensation in the RGB color space, tailored to different water conditions and lighting scenarios, and utilizes this condition to guide the denoising process. In the experimental section, the proposed DiffWater method is tested on four real underwater image datasets and compared against existing methods. Experimental results demonstrate that DiffWater outperforms existing comparison methods in terms of enhancement quality and effectiveness, exhibiting stronger generalization capabilities and robustness.

Abstract Coloured point cloud (CPC) is one of the important representations of three‐dimensional objects, which has been used in many fields. CPC may encounter geometric and colour distortion during its compression, simplification or other processing. Thus, the objective visual quality assessment of CPC is one of the urgent issues to be resolved in the CPC's applications. Aiming at this problem, this paper proposes a new full‐reference visual quality assessment metric for CPC based on geometric segmentation and colour transformation (CPC‐GSCT), which analyzes geometric distortion and colour distortion of CPC. First, considering the visual masking effect of CPC's geometric information, CPC is segmented into different regions and distributed with different weights to describe the influence of visual masking effect in CPC quality assessment. At the same time, a geometric combination feature vector is defined and extracted for measuring the CPC's geometric distortion. Then, considering the colour perception of human eyes, a colour combination feature vector is extracted to measure the CPC's colour distortion in HSV colour space. Finally, all the extracted geometric and colour features are constituted as a feature vector to predict the quality of CPC. Experimental results on three databases (IRPC, SJTU‐PCQA and CPCD2.0) show that the proposed CPC‐GSCT metric can achieve better performance in predicting the visual quality of CPC than relevant existing methods.

In contrast with traditional images, omnidirectional image (OI) has a higher resolution and provides the user with an interactive wide field of view. OI with equirectangular projection (ERP) format, as the default for encoding and transmitting omnidirectional visual contents, is not suitable for quality assessment of OI because of serious geometric distortion in the bipolar regions, especially for blind image quality assessment. In this paper, a segmented spherical projection (SSP) based blind omnidirectional image quality assessment (SSP-BOIQA) method is proposed. The OI with ERP format is first converted into that with SSP format, so as to solve the problem of stretching distortion in the bipolar regions of ERP format, but retain the equatorial region of ERP format. On the one hand, considering that the bipolar regions of the SSP format are circular, a local/global perceptual features extraction scheme with fan-shaped window is proposed for estimating the distortion in the bipolar regions of OI. On the other hand, the perceptual features of the equatorial region are extracted with heat map as weighting factor to reflect users' visual behavior. Then, the features extracted from the OI's bipolar and equatorial regions are pooled to predict the quality of distorted OIs. The experiments on two databases, namely CVIQD2018 and MVAQD databases, demonstrate that the proposed SSP-BOIQA method outperforms the state-of-the-art blind quality assessment methods, and is more consistent with human visual perception.

Cell-phone use while driving results in potentially severe safety hazards. In this paper, a scheme for detecting cell-phone use that is based on deep learning is proposed, which can eliminate the potential risk by detecting the driver behavior and issuing an early warning. The proposed scheme consists of two stages: model training and practical testing. In the former, a multi-angle arrangement of cameras is first designed. Then, based on self-established data set, two independent convolutional neural networks (CNNs) are trained by optimizing the size and number of the convolution kernels, which can efficiently recognize cell-phones and hands in real time. In the testing stage, dynamic region extraction and skin color detection are employed as preprocessing to improve the accuracy of target recognition. Then, with the trained CNNs, the detection of cell-phone and hand targets is carried out, and the corresponding early warning is issued based on the distance of the interaction between the cell-phone and the hand. Numerous experiments are conducted and the results demonstrate that the proposed scheme can accurately detect cell-phone use during driving in real time, with a running time of 144 fps and an accuracy of 95.7%.

A high dynamic range (HDR) stereoscopic omnidirectional vision system can provide users with more realistic binocular and immersive perception, where the HDR stereoscopic omnidirectional image (HSOI) suffers distortions during its encoding and visualization, making its quality evaluation more challenging. To solve the problem, this paper proposes a client-oriented blind HSOI quality metric based on visual perception. The proposed metric mainly consists of a monocular perception module (MPM) and binocular perception module (BPM), which combine monocular/binocular, omnidirectional and HDR/tone-mapping perception. The MPM extracts features from three aspects: global color distortion, symmetric/asymmetric distortion and scene distortion. In the BPM, the binocular fusion map and binocular difference map are generated by joint image filtering. Then, brightness segmentation is performed on the binocular fusion image, and distinctive features are extracted on the segmented high/low/middle brightness regions. For the binocular difference map, natural scene statistical features are extracted by multi-coefficient derivative maps. Finally, feature screening is used to remove the redundancy between the extracted features. Experimental results on the HSOID database show that the proposed metric is generally better than the representative quality metric, and is more consistent with the subjective perception.

Aiming at the shortcomings of the coding rate distortion model in traditional video coding,considering the characteristics of equal rectangular omnidirectional video,a method of omnidirectional video coding rate distortion optimization based on spherical weighted structural similarity (WS-SSIM) was proposed.By considering the distortion of the internal structural similarity of the projection plane and the relationship between the spherical distortion and the projection plane distortion,the “spherical weighted structural similarity” was proposed to describe the degree of distortion of the planar omnidirectional image local block relative to the viewing sphere,which was applied to the rate-distortion optimization process of omnidirectional video coding and adaptive selection of quantization parameters to improve vision-based coding efficiency.The experimental results show that compared with the HEVC video coding standard HM16.9 test platform,the proposed method can save an average of 24.48% code rate under the same visual quality,which indicates that the method has significant performance for improving the omnidirectional video coding rate distortion performance.

High efficiency video coding (HEVC) has high computational complexity,and fast algorithm cannot perform video coding under restricted coding time.Therefore,a complexity control method of HEVC based on random forest was proposed.Firstly,three random forest classifiers with different prediction accuracy were trained to provide various coding configurations for coding tree unit (CTU).Then,an average depth-complexity model was built to allocate CTU complexity.Finally,the CTU coding configuration,determined by the smoothness,average depth,bit,and CTU-level accumulated coding error,was used to complete complexity control.The experimental results show that the proposed method has better complexity control precision,and outperforms the state-of-the-art method in terms of video quality.

Tone mapping(TM) aims to adapt high dynamic range (HDR) images to conventional displays with visual information preserved. In this paper, a novel TM method based on macro-micro modeling is proposed, which can address the common problems in existing TM methods, such as exposure imbalance and halo artifact. From a microscopic perspective, multi-layer decomposition and reconstruction are applied to model the properties of brightness, structure, and detail for HDR images, and then different strategies are adopted for each layer by the human visual system (HVS) to reduce the overall brightness contrast and retain as much scene information. From a macroscopic perspective, scene content-based global operator is designed to adaptively adjust the scene brightness so that it is consistent with the subjective perception of human eyes. Both the micro and macro models are processed in parallel, which can ensure the integrity and subjective consistency of scene information. Experiments with numerous HDR images and TM methods are conducted and the results show that the proposed method achieves visually compelling results with little exposure imbalance and halo artifact, and is superior to the current state-of-the-art TM methods in both subjective and objective evaluations.

High dynamic range (HDR) imaging technique has received much attentions in the recent years for its abundant details and wide dynamic range of luminance. In order to display HDR images on low dynamic range display devices, tone mapping (TM) process is required. However, the TM can be deemed as an inevitable attack in the field of HDR image copyright protection. In this paper, a highly robust HDR image watermarking algorithm based on Tucker decomposition is proposed. Firstly, Tucker decomposition is carried out on HDR color image to obtain the first feature map of the core tensor, which contains most of the energies in the HDR host image. Then, a Auto-Regressive prediction method is used to establish a local correlation model of the first feature map, so that watermark can be embedded in the first feature map according to the prediction result and its true value. Furthermore, a low-complexity luminance masking approach is designed based on modified specular free map to exclude the areas not suitable for watermark embedding, and an embedding intensity selection strategy is used to balance the imperceptibility and robustness of the watermark. The experimental results show that the proposed algorithm is robust against existing TM attacks while keeping better image imperceptibility and embedding capacity.

Abstract The next-generation High Efficiency Video Coding (HEVC) standard reduces the bit rate by 44% on average compared to the previous-generation H.264 standard, resulting in higher encoding complexity. To achieve normal video coding in power-constrained devices and minimize the rate distortion degradation, this paper proposes a hierarchical complexity control algorithm for HEVC on the basis of the coding unit depth decision. First, according to the target complexity and the constantly updated reference time, the coding complexity of the group of pictures layer and the frame layer is allocated and controlled. Second, the maximal depth is adaptively assigned to the coding tree unit (CTU) on the basis of the correlation between the residual information and the optimal depth by establishing the complexity-depth model. Then, the coding unit smoothness decision and adaptive low bit threshold decision are proposed to constrain the unnecessary traversal process within the maximal depth assigned by the CTU. Finally, adaptive upper bit threshold decision is used to continue the necessary traversal process at a larger depth than the maximal depth of allocation to guarantee the quality of important coding units. Experimental results show that our algorithm can reduce the encoding time by up to 50%, with notable control precision and limited performance degradation. Compared to state-of-the-art algorithms, the proposed algorithm can achieve higher control accuracy.

Abstract Asymmetric stereoscopic video coding is becoming increasingly popular, as it can reduce the bandwidth required for stereoscopic 3D delivery without degrading the visual quality. Based on the perceptual theory of binocular suppression, the left and right views of stereoscopic video are coded with different levels of quality. However, existing asymmetric perceptual coding approaches on stereoscopic video mainly focus on the threshold of whole image distortion. It is not so reasonable to use a single unified rather than adaptable perception threshold for a random natural stereoscopic image as the texture complexity typically varies in different blocks of image. In this paper, we generated an asymmetrically distorted stereoscopic image set with different texture densities and conducted a large number of subjective perceptual experiments. A strong correlation between the asymmetrical visibility threshold and texture complexity is revealed from the subjective experiments, and a texture-based asymmetrical visibility threshold model (TAVT) is established. Then, the model is extended to the hierarchical B picture (HBP) coding architecture and an asymmetric stereoscopic video coding method is proposed based on the TAVT model. Experimental results show that the proposed method can effectively reduce the unnecessary perceptual redundancy without visual quality degradation. Especially, it is more efficient for high bitrate configuration.

A quality assessment metric of stereoscopic image in free viewpoint video system was proposed.After the reference and distorted images being decomposed by wavelet then weighted by contrast sensitivity function respectively,the content score was obtained by calculating the average structural similarity.After the center region being found in the binocular matching region,the average phase similarity was calculated on the region to obtain depth perception.The distortion region perceived by the human eye in the matching region by using the threshold,the average structural similarity was calculated between this region and the corresponding region of the other view image by disparity mapping,then the binocular characteristic score was obtained.The final image quality can be obtained by fusing score of each part.The experimental results show that the proposed algorithm has better consistency with subjective perception and it is more in line with human visual characteristics.

In remote medical diagnosis, the percentage of poor-quality fundus images is very high, which requires automated quality assessment of fundus images in the acquisition stage to reduce the retransmission cost. In this paper, we propose a fundus image quality classifier via the analysis of illumination, naturalness, and structure, which use three effective secondary indices (or 5-D feature set) and different classification methods to determine the recommendation indexes of fundus images for further diagnosis. We construct a fundus image database including `accept' and `reject' classes based on the definition of illumination, naturalness, and structure. The model can achieve a sensitivity of 94.69%, specificity of 92.29%, and accuracy of 93.60% for the classifying of the fundus images.

A tone-mapped image (TMI) converted from its high-dynamic-range image (HDRI) tends to appear overexposed in its brightest regions or underexposed in its darkest regions, resulting in inevitable loss of details and impaired naturalness and aesthetics. To address this issue, this paper proposes a novel blind TMI quality assessment (BTMIQA) method for HDRIs used in different dynamic range displays. The brightest and darkest regions of the TMI are first defined and segmented, and their local detail features are used in combination with the global detail feature of the TMI to evaluate the detail distortion. The natural scene statistics features of the luminance and yellow channels of the TMI are then used to evaluate the luminance naturalness and chrominance naturalness, respectively. Subsequently, to predict the aesthetics of the TMI, a series of colorfulness features are extracted and concatenated in a final feature vector, which is used to evaluate the TMI quality by random forest regression. Experiments are performed on the TMI database and ESPL-LIVE high-dynamic-range database, and the results show the effectiveness of the proposed method. Compared with the existing TMI quality assessment methods, the proposed BTMIQA method is more consistent with human visual perception.

Since most of the color image watermarking methods embed the watermark information in each channel or one channel of a color image, the redundant information of the color image cannot be sufficiently utilized, resulting in the poor ability to resist attacks. In this paper, a novel blind color image watermarking method based on the tensor domain is proposed, and it takes efficient account of the overall characteristics of color images and spreads the watermark information into three channels of color images based on tensor decomposition. Exploring the new tensor domain to embed and extract watermark information and the theoretical proof of the optimal embedding position of the core tensor are the major technical contributions of this paper. To be more specific, first, the RGB channels of the color image as a tensor are considered, and the tensor decomposition is used to obtain the core tensor. Then, based on the theoretical proof, the optimal embedding position of the core tensor is obtained, and the watermark information is embedded into the core tensor. Finally, the watermark information is spread to the three channels of the color image through the inverse tensor decomposition. The experimental results show that the proposed method has better invisibility and stronger robustness for the most common attacks.

In existing literature, many tone mapping operators have been designed to convert high dynamic range (HDR) images to low dynamic range (LDR) images for visualization on the standard LDR displays. However, it inevitably causes distortions and artifacts due to the dynamic range compression, resulting in an unpleasant visual experience. In this paper, we propose a no reference (NR) quality evaluation method for tone-mapped (TM) HDR images based on global and local feature analyses. The key strategy of the proposed method lies in measuring the abnormal exposure both globally and locally as well as the halo effects. To be specific, given that the abnormal exposure usually induces the color and brightness changes, the global colorfulness and distribution of brightness are first extracted to quantify the abnormal degree of exposure. Then, since abnormal exposure has diverse influences on local regions, the local features are further extracted from the statistical perspective on the discrete cosine transform domain. Finally, the edge strength is computed locally to measure the halo effects. All extracted quality-sensitive features are combined and trained together with subjective ratings to form a regression model using support vector regression. With extensive experiments, we have shown that the proposed method outmatches several mainstream NR image quality assessment methods designed for both natural scene images and TM HDR images.

Screen content coding (SCC) significantly improves the screen content compression efficiency over the high efficient video coding but at the cost of extremely huge computational complexity. The flexible quad-tree coding unit (CU) partitioning structure and the newly-introduced SCC intra modes are largely responsible for the high computational complexity. In order to meet the challenge of huge computational complexity of SCC, we propose an efficient CU and prediction unit (PU) decision based on neural network (NN) and gray level co-occurrence matrix (GLCM) for SCC intra prediction. The proposed efficient SCC intra prediction algorithm contains three stages, including NN-based CU classification model, efficient PU mode decision based on classification (EPMD), and efficient CU size decision based on GLCM and spatiotemporal correlations (ECSD). Consequently, the computational complexity of SCC intra prediction can be drastically reduced by replacing the brute-force search with EPMD and ECSD to decide the optimal combination of CU size and PU mode. In addition, an online updating method of weighted factor is introduced to cope with different characteristics of test sequences. In order to achieve a good tradeoff between complexity reduction and rate distortion (RD) performance, extensive experiments are conducted to select the optimal threshold for ECSD. The experimental results show that in comparison with the original SCC reference software, the proposed algorithm can reduce 49.33% intra coding time with 1.36% BDBR increase and 0.13 dB BDPSNR decrease. Meanwhile, the proposed algorithm outperforms eight state-of-the-art algorithms in terms of computational complexity reduction and RD performance.

Multiview video plus depth is one of the mainstream representations of 3D scenes in emerging free viewpoint video, which generates virtual 3D synthesized images through a depth-image-based-rendering (DIBR) technique. However, the inaccuracy of depth maps and imperfect DIBR techniques result in different geometric distortions that seriously deteriorate the users’ visual perception. An effective 3D synthesized image quality assessment (IQA) metric can simulate human visual perception and determine the application feasibility of the synthesized content. In this paper, a no-reference IQA metric based on visual-entropy-guided multi-layer features analysis for 3D synthesized images is proposed. According to the energy entropy, the geometric distortions are divided into two visual attention layers, namely, bottom-up layer and top-down layer. The feature of salient distortion is measured by regional proportion plus transition threshold on a bottom-up layer. In parallel, the key distribution regions of insignificant geometric distortion are extracted by a relative total variation model, and the features of these distortions are measured by the interaction of decentralized attention and concentrated attention on top-down layers. By integrating the features of both bottom-up and top-down layers, a more visually perceptive quality evaluation model is built. Experimental results show that the proposed method is superior to the state-of-the-art in assessing the quality of 3D synthesized images.

Although the perception based rate distortion optimization (RDO) and coding algorithms can improve subjective quality,they significantly decrease rate distortion performance.For the above questions,a saliency-based RDO method was proposed.Firstly,according to the relation between rate distribution and saliency features of coding tree unit in HEVC,a priori saliency-rate model was designed.Furthermore,the initial parameters of the model were determined.Secondly,the rate difference minimization rate distortion model was proposed,and then a calculation method of the adaptive Lagrangian multiplier was derived from the model.Finally,update mechanism of model parameters was designed.Experimental results show that the proposed algorithm can improve the subjective quality while it maintains rate distortion performance.

Quality assessment of 3-D images encounters more challenges in better understanding of human visual system. In this paper, we propose a perceptual quality assessment approach for stereoscopic images by modeling visual properties of the primary visual cortex. For this purpose, we obtain a new feature encoding approach for the visual information, and define a new similarity measure approach to match the feature encoding to give more reliable and accurate quality assessment. Experimental results on three symmetrically, asymmetrically, and multiple distorted stereoscopic image quality assessment databases demonstrate that our method has high consistency with subjective assessment.

Neighborhood selection is very important for local region feature learning in point cloud learning networks. Different neighborhood selection schemes may lead to quite different results for point cloud processing tasks. The existing point cloud learning networks mainly adopt the approach of customizing the neighborhood, without considering whether the selected neighborhood is reasonable or not. To solve this problem, this paper proposes a new point cloud learning network, denoted as Dynamic neighborhood Network (DNet), to dynamically select the neighborhood and learn the features of each point. The proposed DNet has a multi-head structure which has two important modules: the Feature Enhancement Layer (FELayer) and the masking mechanism. The FELayer enhances the manifold features of the point cloud, while the masking mechanism is used to remove the neighborhood points with low contribution. The DNet can learn the manifold features and spatial geometric features of point cloud, and obtain the relationship between each point and its effective neighborhood points through the masking mechanism, so that the dynamic neighborhood features of each point can be obtained. Experimental results on three public datasets demonstrate that compared with the state-of-the-art learning networks, the proposed DNet shows better superiority and competitiveness in point cloud processing task.

With the wide applications of three-dimensional (3D) meshes in intelligent manufacturing, digital animation, virtual reality, digital cities and other fields, more and more processing techniques are being developed for 3D meshes, including watermarking, compression, and simplification, which will inevitably lead to various distortions. Therefore, how to evaluate the visual quality of 3D mesh is becoming an important problem and it is necessary to design effective tools for blind 3D mesh quality assessment. In this paper, we propose a new Blind Mesh Quality Assessment method based on Graph Spectral Entropy and Spatial features, called as BMQA-GSES. 3D mesh can be represented as graph signal, in the graph spectral domain, the Gaussian curvature signal of the 3D mesh is firstly converted with Graph Fourier transform (GFT), and then the smoothness and information entropy of amplitude features are extracted to evaluate the distortion. In the spatial domain, four well-performing spatial features are combined to describe the concave and convex information and structural information of 3D meshes. All the extracted features are fused by the random forest regression to predict the objective quality score of the 3D mesh. Experiments are performed successfully on the public databases and the obtained results show that the proposed BMQA-GSES method provides good correlation with human visual perception and competitive scores compared to state-of-art quality assessment methods.

In order to overcome the poor robustness of traditional image registration algorithms in illuminating and solving the problem of low accuracy of a learning-based image homography matrix estimation algorithm, an image registration algorithm based on convolutional neural network (CNN) and local homography transformation is proposed. Firstly, to ensure the diversity of samples, a sample and label generation method based on moving direct linear transformation (MDLT) is designed. The generated samples and labels can effectively reflect the local characteristics of images and are suitable for training the CNN model with which multiple pairs of local matching points between two images to be registered can be calculated. Then, the local homography matrices between the two images are estimated by using the MDLT and finally the image registration can be realized. The experimental results show that the proposed image registration algorithm achieves higher accuracy than other commonly used algorithms such as the SIFT, ORB, ECC, and APAP algorithms, as well as another two learning-based algorithms, and it has good robustness for different types of illumination imaging.

Because perceptual video coding (PVC) can reduce bitrates with negligible visual quality loss in video compression, a PVC scheme based on just noticeable distortion (JND) model is proposed for ultra-high definition video. Firstly, the proposed JND model is designed, considering the spatial JND characteristics such as contrast sensitivity, luminance adaptation and saliency weight factor. Secondly, in order to perform precise JND suppression, the Gauss differential entropy (GDE) filter is designed to divide the image into smooth and complex texture region. Thirdly, through incorporating the proposed JND model into the encoding process, the transform coefficients are suppressed in harmonization with the transform/quantization process of high efficiency video coding (HEVC). In order to achieve the JND suppression effectively, a distortion compensation factor and distortion compensation control factor are incorporated to control the extent of distortion in the rate distortion optimization process. The experimental results show that the proposed PVC scheme can achieve a remarkable bitrate reduction of 32.98% for low delay (LD) configuration and 28.61% for random access (RA) configuration with a negligible subjective quality loss. Meanwhile, the proposed method only causes about average 12.94% and 22.45% encoding time increase under LD and RA configuration compared with an HEVC reference software, respectively.

Well-performed Video quality assessment (VQA) method should be consistent with human visual systems for better prediction accuracy. In this paper, we propose a VQA method using motion-compensated temporal filtering (MCTF) and manifold feature similarity. To be more specific, a group of frames (GoF) is first decomposed into a temporal high-pass component (HPC) and a temporal low-pass component (LPC) by MCTF. Following this, manifold feature learning (MFL) and phase congruency (PC) are used to predict the quality of temporal LPC and temporal HPC respectively. The quality measures of the LPC and the HPC are then combined as GoF quality. A temporal pooling strategy is subsequently used to integrate GoF qualities into an overall video quality. The proposed VQA method appropriately processes temporal information in video by MCTF and temporal pooling strategy, and simulate human visual perception by MFL. Experiments on publicly available video quality database showed that in comparison with several state-of-the-art VQA methods, the proposed VQA method achieves better consistency with subjective video quality and can predict video quality more accurately.

Due to sharp changes in local brightness in high dynamic range scenes, fused images obtained by the traditional multi-exposure fusion methods usually have an unnatural appearance resulting from halo artifacts. In this paper, we propose a halo-free multi-exposure fusion method based on sparse representation of gradient features for high dynamic range imaging. First, we analyze the cause of halo artifacts. Since the range of local brightness changes in high dynamic scenes may be far wider than the dynamic range of an ordinary camera, there are some invalid, large-amplitude gradients in the multi-exposure source images, so halo artifacts are produced in the fused image. Subsequently, by analyzing the significance of the local sparse coefficient in a luminance gradient map, we construct a local gradient sparse descriptor to extract local details of source images. Then, as an activity level measurement in the fusion method, the local gradient sparse descriptor is used to extract image features and remove halo artifacts when the source images have sharp local changes in brightness. Experimental results show that the proposed method obtains state-of-the-art performance in subjective and objective evaluation, particularly in terms of effectively eliminating halo artifacts.

Digital watermarking is an effective way for authentication of stereo images when they are transmitted over networks. Since the left and right images of a stereo image pair are not independent but similar to each other, their watermarking design should be different from that of mono-images. This paper thus proposes a stereo matching based stereo image watermarking scheme for tamper detection and recovery. One-to-one mappings between left and right image blocks are established, and tamper detection bits generated from each block and the recovery bits of the mapped block are embedded back to the block itself. To keep the consistency between left and right images, a stereo image pair is usually tampered symmetrically. That is, the tampered areas in left and right images are similar to each other, thus the combination of tamper detection results of left and right images can facilitate more accurate tamper detection results. In general, a tampered block in one image of stereo image pair can be recovered with the recovery bits embedded in its mapped block in the other view. However, if the mapped block is also tampered, stereo matching based method is employed to recover the tampered block. Our experimental results show that the proposed tamper detection scheme can identify the tampered blocks with the probability of almost 1 if the stereo image is modified symmetrically. Moreover, the proposed scheme is able to recover a view of stereo images altered 100% which may even confuse the human eyes.

To compress stereo video effectively, this paper proposes a novel macroblock (MB) level rate control method based on binocular perception. A binocular just-notification difference (BJND) model based on the parallax matching is first used to describe binocular perception. Then, the proposed rate control method is performed in stereo video coding with four levels, namely, view level, group-of-pictures (GOP) level, frame level, and MB level. In the view level, different proportions of bitrates are allocated for the left and right views of stereo video according to the prestatistical rate allocation proportion. In the GOP level, the total number of bitrates allocated to each GOP is computed and the initial quantization parameter of each GOP is set. In the frame level, the target bits allocated to each frame are computed. In the MB level, visual perception factor, which is measured by the BJND value of MB, is used to adjust the MB level bit allocation, so that the rate control results in line with the human visual characteristics. Experimental results show that the proposed method can control the bitrate more accurately and get better subjective quality of stereo video, compared with other methods.

We present a simple quality assessment index for stereoscopic images based on 3D gradient magnitude. To be more specific, we construct 3D volume from the stereoscopic images across different disparity spaces and calculate pointwise 3D gradient magnitude similarity (3D-GMS) along three horizontal, vertical, and viewpoint directions. Then, the quality score is obtained by averaging the 3D-GMS scores of all points in the 3D volume. Experimental results on four publicly available 3D image quality assessment databases demonstrate that, in comparison with the most related existing methods, the devised algorithm achieves high consistency alignment with subjective assessment.

We propose a Stereoscopic Visual Attention- (SVA-) based regional bit allocation optimization for Multiview Video Coding (MVC) by the exploiting visual redundancies from human perceptions. We propose a novel SVA model, where multiple perceptual stimuli including depth, motion, intensity, color, and orientation contrast are utilized, to simulate the visual attention mechanisms of human visual system with stereoscopic perception. Then, a semantic region-of-interest (ROI) is extracted based on the saliency maps of SVA. Both objective and subjective evaluations of extracted ROIs indicated that the proposed SVA model based on ROI extraction scheme outperforms the schemes only using spatial or/and temporal visual attention clues. Finally, by using the extracted SVA-based ROIs, a regional bit allocation optimization scheme is presented to allocate more bits on SVA-based ROIs for high image quality and fewer bits on background regions for efficient compression purpose. Experimental results on MVC show that the proposed regional bit allocation algorithm can achieve over 20∼30% bit-rate saving while maintaining the subjective image quality. Meanwhile, the image quality of ROIs is improved by 0.46∼0.61 dB at the cost of insensitive image quality degradation of the background image.

Multiview video coding (MVC) plays an important role in three-dimensional video applications. Joint Video Team developed a joint multiview video model (JMVM) in which full-search algorithm is employed in macroblock mode selection to provide the best rate distortion performance for MVC. However, it results in a considerable increase in encoding complexity. We propose a hybrid fast macroblock mode selection algorithm after analyzing the full-search algorithm of JMVM. For nonanchor frames of the base view, the proposed algorithm halfway stops the macroblock mode search process by designing three dynamic thresholds. When nonanchor frames of the other views are being encoded, the macroblock modes can be predicted from the frames of the neighboring views due to the strong correlations of the macroblock modes. Experimental results show that the proposed hybrid fast macroblock mode selection algorithm promotes the encoding speed by 2.37 ∼ 9.97 times without noticeable quality degradation compared with the JMVM.