Search

With the developing explosion in video data delivery, perceptual video coding (PVC) plays an increasingly significant role in video compression. The just noticeable distortion (JND) reflects the tolerance limit of human visual system (HVS) to coding distortion directly, resulting in that JND-based PVC is the most important branch of video coding. This paper provides an extensive overview of JND estimation and JND-based PVC, so as to make interested readers aware of the status quo. The main contribution of this article can be briefly outlined as follows. Firstly, the general description of JND concepts and most existing computation models for JND are to be reviewed systematically. Secondly, most related works about JND-based perceptual image and video coding schemes are introduced, including JND-based coding preprocessing and JND-based codec embedding. Thirdly, in addition to a thorough summary of JND estimation and JND-based PVC, possible future directions and opportunities are analyzed and discussed. • Firstly, the general description of JND concepts and most existing computation models for JND are to be reviewed systematically. • Secondly, most related works about JND-based perceptual image or video coding schemes are introduced, including JND-based coding preprocessing and JND-based codec embedding. • Thirdly, in addition to a thorough summary of JND estimation and JND-based PVC, possible future directions and opportunities are analyzed and discussed. [ABSTRACT FROM AUTHOR]

In visual perception, human only perceive discrete-scale quality levels over a wide range of coding bitrate. More clearly, the videos compressed with a series of quantization parameters (QPs) only have limited perceived quality levels. In this paper, perceptual quantization is transformed into the problem of how to determine the just perceived QP for each quality level, and a just noticeable coding distortion (JNCD) based perceptual quantization scheme is proposed. Specifically, multiple visual masking effects are analyzed and a linear regression (LR) based JNCD model is proposed to predict JNCD thresholds for all quality levels at first. According to the JNCD prediction model, the frame-level perceptual QPs for all quality levels are then derived on the premise of that coding distortions are infinitely close to the predicted JNCD thresholds. Based on the predicted frame-level perceptual QPs, the perceived QPs of all quality levels for each coding unit (CU) are finally determined according to a perceptual modulation function. Experimental results show that the proposed quality-wise perceptual quantization scheme is superior to the existing perceptual video coding algorithms significantly, i.e., the proposed perceptual quantization could save more bitrate with better quality. [ABSTRACT FROM AUTHOR]

• Images blocks are classified based on the MAD-based block classification method. • The contrast intensity of the block is measured by the MAD. • A disorder evaluation metric is proposed to measure the disorder of the block. • The disorderly concealment effect is incorporated into JND estimation. The just noticeable distortion (JND) in the contour and orderly regions is easy to be overestimated and that in the disorderly areas is usually underestimated. In order to estimate the JND threshold more accurately, this paper proposes an improved DCT-based JND estimation model considering multiple masking effects properly. The contributions of this paper are characterized by twofold. On the one hand, a mean absolute difference based (MAD-based) block classification method is developed at first to classify image blocks into plain, contour and texture types accurately and quickly. And the JND model for contrast masking effect (CM-JND) is constructed as a modulation factor based on the MAD of each block. On the other hand, we propose a distance-based disorder evaluation metric to measure the disorder intensity in block level. Then, the JND model for the disorderly concealment effect (DC-JND) is proposed based on our psychological experiment. Finally, the total JND estimation threshold is modeled by fusing the spatial contrast sensitivity function, the luminance adaptation effect, the CM and DC effects. Experimental results show that the proposed DCT-based JND estimation model outperforms existing models in performance and complexity. Specifically, the proposed model shows more tolerance for distortions, lower computational complexity with better perceptual quality than other JND models. [ABSTRACT FROM AUTHOR]

With the rapid development of the Internet and multimedia technologies, multimedia applications integrating audio and video are becoming increasingly prevalent in both everyday life and professional environments. A critical challenge is to significantly enhance compression efficiency and bandwidth utilization while maintaining high-quality user experiences. To address this challenge, the Just Noticeable Distortion (JND) estimation model, which leverages the perceptual characteristics of the Human Visual System (HVS), is widely used in image and video coding for improved data compression. However, human visual perception is an integrative process that involves both visual and auditory stimuli. Therefore, this paper investigates the influence of audio signals on visual perception and presents a collaborative audio–video JND estimation model tailored for multimedia applications. Specifically, we characterize audio loudness, duration, and energy as temporal perceptual features, while assigning the audio saliency superimposed on the image plane as the spatial perceptual feature. An audio JND adjustment factor is then designed using a segmentation function. Finally, the proposed model combines the video-based JND model with the audio JND adjustment factor to form the audio–video collaborative JND estimation model. Compared with existing JND models, the model presented in this paper achieves the best subjective quality, with an average PSNR value of 26.97 dB. The experimental results confirm that audio significantly impacts human visual perception. The proposed audio–video collaborative JND model effectively enhances the accuracy of JND estimation for multimedia data, thereby improving compression efficiency and maintaining high-quality user experiences. • An accurate just noticeable distortion estimation model facilitates data compression. • Audio features in multimedia applications can influence visual perception. • Audio feature extraction is categorized into temporal and spatial domains. • Audiovisual collaboration enhances the accuracy of the just noticeable distortion. [ABSTRACT FROM AUTHOR]

Due to the high resolution of 360-degree video, it needs more time to compress using the latest Versatile Video Coding (VVC) compared to traditional video. Meanwhile, the introduction of quad-tree plus multi-type tree (QTMT) in VVC enhances compression performance, but the coding complexity also increases significantly. This paper proposes a fast intra coding unit (CU) partition algorithm for 360-degree video. First, we create a large-scale CU partition mode (CPM) dataset. Next, we design an adaptive CU partition decision network (ACPD-Net) based on adaptive QP, which can simultaneously predict the optimal CPM of four 32 × 32 sub-CUs from the global texture information of 64 × 64 CU. Then, based on the partitioning characteristics of the mapped 360-degree video, the partition structure of 32 × 32 CU is simplified from QTMT to QTBT. The experimental findings indicate that the proposed method can decrease the complexity of intra-frame coding by 61.10% and increase BDBR by 1.99% on average. [ABSTRACT FROM AUTHOR]

There are many challenging scenarios in the task of salient object detection in optical remote sensing images (RSIs), such as various scales and irregular shapes of salient objects, cluttered backgrounds, etc. Therefore, it is difficult to directly apply saliency models targeting natural scene images to optical RSIs. Besides, existing models often do not give sufficient exploration for the potential relationship of different salient objects or different parts of the salient object. In this paper, we propose a graph interaction network (i.e. GINet) with semantic-guided spatial refinement to conduct salient object detection in optical RSIs. The key advantages of GINet lie in two points. Firstly, the graph interactive reasoning (GIR) module conducts information exchange of different-level features via the graph interaction operation, and enhances features along spatial and channel dimensions via the graph reasoning operation. Secondly, we designed the global content-aware refinement (GCR) module, which incorporates the foreground and background feature-based local information and the semantic feature-based global information simultaneously. Experiments results on two public optical RSIs datasets clearly show the effectiveness and superiority of the proposed GINet when compared with the state-of-the-art models. • Design a graph interactive network with Semantic-Guided for spatial refinement. • Perform graph interaction to promote the information exchange. • Conduct graph reasoning along the spatial and channel dimensions to elevate features. • Integrates the foreground and background cues-based local and global information. [ABSTRACT FROM AUTHOR]

• A novel fast intra mode decision algorithm is designed for VVC. • Candidate mode list is reduced based on the correlation between the RMD modes and MPMs. • Early termination of RDO process is performed based on the correlation between RMD cost and RDO cost. The latest video coding standard Versatile Video Coding (VVC) obtains superior coding efficiency compared to the High Efficiency Video Coding (HEVC), which is achieved by incorporating more effective and complex new coding tools. In this paper, we propose a novel fast intra mode decision algorithm for VVC, including following two strategies: (1) the correlation between the optimal modes of the adjacent blocks and the modes selected in the rough modes decision (RMD) process is analyzed and applied to reduce the modes in the candidate list; (2) modes in the candidate list are sorted in ascending order according to the modes' cost calculated in the RMD process. An early termination method is proposed for terminating the optimal prediction mode decision process based on this new order early. These two strategies are incorporated into intra coding to reduce the coding complexity. Since these two strategies do not add any additional computational complexity, the proposed fast algorithm can achieve more complexity reduction. The experimental results show that the complexity reduction of the proposed algorithm is up to 44.74% compared to VVC reference software VTM2.0, and averagely 30.59% encoding time saving with 0.86% BDBR increase. [ABSTRACT FROM AUTHOR]

Matrix metalloproteinases (MMPs) are important contributing factors of tissue remodeling and wound healing. MMP9, a predominant soluble MMP, has been discovered as one of the most up-regulated genes in peripheral nerves after nerve injury, implying the potential regulatory roles of MMP9 during peripheral nerve regeneration. Considering that Schwann cell is a main cell population in peripheral nerves and MMP9 is secreted by Schwann cells, here, we investigated the biological functions of MMP9 on Schwann cell phenotype modulation. MMP9 gene knockdown or MMP9 recombinant protein exposure significantly hinders or elevates the migration ability of cultured Schwann cells, respectively. Direct application of MMP9 recombinant protein to sciatic nerve injured rats promotes Schwann cell migration, blood vessel formation, axon elongation, and myelin wrapping. Genetic exploration of MMP9-induced changes indicates that MMP9 regulates the extracellular region as well as the intracellular metabolism of Schwann cells. Our present study illuminates the importance of elevated MMP9 after nerve injury from the functional aspect and enhances our comprehension of the mechanisms underlying peripheral nerve regeneration. • Up-regulation of MMP9 expression after injury plays an important role in peripheral nerve regeneration. • MMP9 advances Schwann cell migration, axon elongation, and myelin wrapping. • Promoting angiogenesis is one of the important factors that MMP9 facilitates peripheral nerve regeneration. [ABSTRACT FROM AUTHOR]

Compared with deadzone hard-decision quantization (HDQ), rate-distortion optimized quantization (RDOQ) in HEVC brings non-negligible coding gain, however consumes considerable computations caused by exhaustive search over multiple candidates to determine optimal output level. Benefiting from efficient prediction in HEVC, transform blocks are frequently quantized to all zero, especially in small-size blocks. It is worthwhile to detect all zero block (AZB) for transform blocks to bypass subsequent computation-intensive RDOQ. Traditional thresholding based AZB detection algorithms are well-suited for deadzone quantized blocks, however miss partial optimal results in RDOQ and suffer from more or less accuracy degradation in RDOQ. This paper proposes a novel multi-stage AZB detection algorithm for RDOQ blocks with good tradeoff between complexity and accuracy. At the first stage, genuine all zero blocks (G_AZB) which are quantized to all zero both in HDQ and RDOQ are prejudged by comparison with conservative threshold determined by mathematical derivation for deadzone HDQ. At the second stage, an adaptive threshold model is built using adaptive deadzone offset by simulating the behavior patterns existing in RDOQ, aiming to further detect the pseudo AZB (P_AZB) which are quantized to all zero in RDOQ however not all zero in HDQ. At the final stage, machine learning based detection is proposed to classify the remaining "cunning" all zero blocks using eight distinguished RDO-related features, by which subtle working mechanism in RDOQ is leveraged. The experimental results demonstrate that the proposed algorithm achieves up to 7.471% total coding computation saving with 0.064% BD-RATE increment compared with RDOQ on average. Moreover, the average FNR and FPR detection accuracies are 6.3% and 6.5% respectively. [ABSTRACT FROM AUTHOR]