Search

The new generation versatile video coding (VVC) standard brings extremely high compression efficiency. Meanwhile, the complexity of the encoder is also greatly increased. In view of this, this paper proposes a fast algorithm for all zero block detection to speed up the quantization process, thereby reducing the complexity of the encoder. The method proposed in this paper consists of three parts. Firstly, genuine all zero blocks are detected based on a fixed threshold which is derived by hard decision quantization formula. Secondly, parts of pseudo all zero blocks are detected based on an adaptive threshold which is derived by analyzing the positions of transform coefficients under a certain condition. Finally, for the remaining pseudo all zero blocks, machine learning is introduced and the decision is made through the fully connected neural network. Experimental results show that the proposed fast algorithm achieves up to 7.505% and 7.049% coding time saving under Low Delay B and Random Access configurations with only 0.470% and 0.578% performance loss on average, respectively. [ABSTRACT FROM AUTHOR]

Constant bit rate (CBR) videos are widely used in streaming playback applications. However, the image quality of the CBR video is often unstable, especially for scenes with large motion. To this end, we design a new model to represent the distortion of High Efficiency Video Coding (HEVC) constant bit rate video, and propose a neural network for a constant bit rate video quality enhancement (CBREN). We propose a dual-domain restoration module (DRM) to jointly learn the prior knowledge in the pixel domain and the frequency domain. To address the degradation resulting from compression, we propose a two-step quantization degradation estimation strategy. The Inverse DCT (IDCT) Translation Unit (ITU) is used to constrain the quantization table of the constant bit rate video to a suitable range, and the Dynamic Alpha Unit (DAU) is used to fine-tune the quantization table according to the content of each frame. In order to effectively reduce the block distortion of different sizes produced in the compression process, we adopt a multi-scale network. Extensive experiments show that our approach can greatly enhance the quality of CBR compressed video. Moreover, our method can also be applied to constant quantization parameter (CQP) video enhancement tasks, and is certainly superior to existing methods. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) provides superior coding efficiency than previous video coding standards at the cost of increasing encoding complexity. The complexity increase of motion estimation (ME) procedure is rather significant, especially when considering the complicated partitioning structure of HEVC. To fully exploit the coding efficiency brought by HEVC requires a huge amount of computations. In this paper, we analyze the ME structure in HEVC and propose a parallel framework to decouple ME for different partitions on many-core processors. Based on local parallel method (LPM), we first use the directed acyclic graph (DAG)-based order to parallelize coding tree units (CTUs) and adopt improved LPM (ILPM) within each CTU (DAGILPM), which exploits the CTU-level and prediction unit (PU)-level parallelism. Then, we find that there exist completely independent PUs (CIPUs) and partially independent PUs (PIPUs). When the degree of parallelism (DP) is smaller than the maximum DP of DAGILPM, we process the CIPUs and PIPUs, which further increases the DP. The data dependencies and coding efficiency stay the same as LPM. Experiments show that on a 64-core system, compared with serial execution, our proposed scheme achieves more than 30 and 40 times speedup for $1920\times 1080$ and $2560\times 1600$ video sequences, respectively. [ABSTRACT FROM AUTHOR]

High‐efficiency video coding (HEVC) is the next generation standard of video coding. The deblocking filter (DF) constitutes a significant part of the HEVC decoder complexity. A three‐step parallel framework (TPF) is proposed for the H.264/AVC DF, which is also suitable for HEVC except the third step. The third step of the TPF is replaced with a directed acyclic graph‐based order. Experiments show that the proposed method dramatically accelerates more than the state‐of‐the‐art parallel method. [ABSTRACT FROM AUTHOR]

As the latest generation of video coding standards, versatile video coding (VVC) introduces several new coding tools in transform coding to concentrate the energy of residual blocks. In this work, we propose a regular multiplier (RM) based hardware architecture for 2D transform that can process different transform sizes and types using a unified architecture. This architecture consists of a 1D row transform, a transpose memory, and a 1D column transform, allowing for processing 16 coefficients per cycle. For the 1D row/column transform, the unified calculation structure consists of 512 regular multipliers supporting various transform sizes and types. For the transpose memory, we design an SRAM-based diagonal storage approach, along with a FIFO for storing the block information. Compared with the state-of-the-art MCM-based design, the proposed RM-based design shows a 30.9% reduction in area while can operate at a higher frequency. [ABSTRACT FROM AUTHOR]

Salient object detection methods based on two-modal images have achieved remarkable success with the aid of image acquisition equipment. However, environmental factors often interfere with the Depth and Thermal maps, rendering them ineffective in providing object information. To address this weakness, we utilize the VDT dataset, which includes Visible, Depth, and Thermal images, and propose a triple-modal interaction encoder and multi-scale fusion decoder network (TMNet) to highlight the salient regions. The triple-modal interaction encoder comprises the separation context-aware feature module, channel-wise fusion module, and triple-modal refinement and fusion module, enabling us to fully explore and utilize the complementarity between Visible, Depth, and Thermal information. The multi-scale fusion decoder involves the semantic-aware localizing module and contour-aware refinement module to extract and fuse the location and boundary information, yielding a high-quality saliency map. Extensive experiments on the public VDT-2048 dataset demonstrate that our TMNet outperforms existing state-of-the-art methods in terms of all evaluation metrics. • This paper proposes a triple-modal interaction encoder and multi-scale fusion decoder network (TMNet) to highlight the salient regions. • The triple-modal interaction encoder comprises the separation context-aware feature module, channel-wise fusion module, and triple-modal refinement and fusion module to explore and utilize the complementarity between Visible, Depth, and Thermal information. • The multi-scale fusion decoder involves the semantic-aware localizing module and contour-aware refinement module to extract and fuse the location and boundary information, yielding a high-quality saliency map. • Extensive experiments on the public VDT-2048 dataset demonstrate that our TMNet outperforms existing state-of-the-art methods in terms of all evaluation metrics. [ABSTRACT FROM AUTHOR]

Affine motion estimation (AME) which is newly introduced in versatile video coding (VVC) plays a significant role in the bit-rate reduction for rotation and zooming scenes. However, it brings the complexity extremely increased. In this paper, a fast AME algorithm is designed to cope with the high-complexity problem. Three crucial stages are involved in the fast algorithm, which is the optimal inter-mode decision based on the coding unit (CU) partition, the fast algorithm in affine motion search (AMS), and the fast inter-mode decision based on a decision tree. Specifically, the optimal inter-mode will be determined straightforwardly when the parent CU is checked whether it is translational motion estimation (TME). Then, three fast algorithms are made in AMS, which are early termination based on control point motion vector (C P M V), early termination of the iteration process, and fast fine granularity C P M V search. Finally, the optimal inter-mode is predicted using a decision tree based on eight essential features. Experimental results show that the proposed algorithm achieves 10.20% and 10.33% encoding time-saving on average under Low Delay B (LDB) and Random Access (RA) configuration, while the BD-Rate loss is only 0.12% and 0.14%, respectively. • Propose a three-stage scheme to decrease the complexity of the affine ME in VVC. • Utilize the CU partition including parent-CU and child-CU to reduce complexity. • Propose a three-step fast algorithm for the fast algorithm in affine motion search. • Find eight key features to predict the optimal inter-mode based on the decision tree model. [ABSTRACT FROM AUTHOR]

This paper introduces a novel video saliency model for salient object detection in videos. Firstly, we generate multi-level deep features via a symmetrical convolutional neural network, in which the inputs are the current frame and the optical flow image. Then, the multi-level deep features are integrated in a hierarchical manner using a fusion network, which deploys attention module to make a selection for deep features. Lastly, the integrated deep feature is combined with the boundary information originated from shallow layer of the feature extraction networks, and the saliency map is generated by the saliency prediction step. The key advantages of our model lie on the attention module, the hierarchical integration and the boundary information, in which the former one acts as weight filter and is used to select the most salient regions in deep features, the middle one gives an effective integration manner for features from different layers and the last one provides well-defined boundaries for saliency map. Extensive experiments are performed on two challenging video dataset, and the experimental results show that our model consistently outperforms the state-of-the-art saliency models in a large margin. [ABSTRACT FROM AUTHOR]

Compared with deadzone hard-decision quantization (HDQ), rate-distortion optimized quantization (RDOQ) in HEVC brings non-negligible coding gain, however consumes considerable computations caused by exhaustive search over multiple candidates to determine optimal output level. Benefiting from efficient prediction in HEVC, transform blocks are frequently quantized to all zero, especially in small-size blocks. It is worthwhile to detect all zero block (AZB) for transform blocks to bypass subsequent computation-intensive RDOQ. Traditional thresholding based AZB detection algorithms are well-suited for deadzone quantized blocks, however miss partial optimal results in RDOQ and suffer from more or less accuracy degradation in RDOQ. This paper proposes a novel multi-stage AZB detection algorithm for RDOQ blocks with good tradeoff between complexity and accuracy. At the first stage, genuine all zero blocks (G_AZB) which are quantized to all zero both in HDQ and RDOQ are prejudged by comparison with conservative threshold determined by mathematical derivation for deadzone HDQ. At the second stage, an adaptive threshold model is built using adaptive deadzone offset by simulating the behavior patterns existing in RDOQ, aiming to further detect the pseudo AZB (P_AZB) which are quantized to all zero in RDOQ however not all zero in HDQ. At the final stage, machine learning based detection is proposed to classify the remaining "cunning" all zero blocks using eight distinguished RDO-related features, by which subtle working mechanism in RDOQ is leveraged. The experimental results demonstrate that the proposed algorithm achieves up to 7.471% total coding computation saving with 0.064% BD-RATE increment compared with RDOQ on average. Moreover, the average FNR and FPR detection accuracies are 6.3% and 6.5% respectively. [ABSTRACT FROM AUTHOR]