Search

Virtual and augmented reality (VR/AR), teleoperation, and telepresence technologies heavily depend on video streaming and playback to enable immersive user experiences. However, the substantial bandwidth requirements and file sizes associated with VR/AR and 360-degree video content present significant challenges for efficient transmission and storage. Modern video coding standards, including HEVC, AV1, VP9, VVC, and EVC, have been designed to address these issues by enhancing coding efficiency while maintaining video quality on par with the H.264 standard. Nonetheless, the adaptive block structures inherent to these video coding standards introduce increased computational complexity, necessitating additional intra-prediction modes. The integration of AI in video coding has the potential to substantially improve video compression efficiency, reduce file sizes, and enhance video quality, making it a crucial area of research and development within the video coding domain. As AI systems can execute a wide array of tasks and adapt to new challenges, their incorporation into video coding may result in even more advanced compression techniques and innovative solutions to meet the ever-evolving demands of the industry. In this study, we introduce a state-of-the-art adaptive instant learning-based model, named iHELP, developed to address the computational complexity arising from encoders' adaptive block structures. The iHELP model achieves outstanding coding efficiency and quality while considerably improving encoding speed. iHELP model has been tested on HEVC, but it applies to other encoders with similar adaptive block structures. iHELP model employs entropy-based block similarity to predict the splitting decision of the LCU, determining whether to divide the block based on the correlation between the block content and previously adjacent encoded blocks in both spatial and temporal dimensions. Our methodology has been rigorously evaluated using the HEVC standard's common test conditions, and the results indicate that iHELP serves as an effective solution for efficient video coding in bandwidth-constrained situations, making it suitable for real-time video applications. The proposed method achieves an 80% reduction in encoding time while maintaining comparable PSNR performance relative to the RDO approach. The exceptional potential of the iHELP model calls for further exploration, as no other existing methods have demonstrated such a high level of performance. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Beijing University of Technology is the property of Journal of Beijing University of Technology, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Screen content denotes images generated by computers, which consists of texts, graphics and synthetic images. Conventional encoding strategies developed for natural content, which are premised on the strong temporal relevance, may not work well for screen content. This is because the irregular motion of objects results in the weak temporal relevance in screen content, which also makes the selection of reference pictures not optimal. This paper proposes an optimal picture reordering scheme for screen content coding to enhance the temporal relevance of screen content. In our scheme, a concept of inter-picture correlation is introduced to help formulate the temporal relevance problem, and a model of permutation is proposed to produce the new order of the pictures. To further improve the coding efficiency, a reference picture selecting algorithm is also proposed. Better compression performance can then be achieved. Compared with SCC extension of HEVC, on average, 2.82% bit rate saving can be achieved by reordering the pictures and 4.40% bit saving is achieved by further utilizing the reference pictures selection algorithm. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) significantly improves the coding efficiency but increases the coding complexity, especially in the process of coding unit (CU) partitioning based on quadtree structure, so it is important to study the fast CU partitioning. A multi-scale feature fusion network can achieve fast HEVC CU partitioning. Therefore, the UcuNet network structure is designed by combining the U-Net and CU partitioning features. Meanwhile, asymmetric convolutional AC and CBAM attention mechanisms are used to enhance the feature extraction of pixels at different scales. In order to sufficiently train the deep learning model, the original video with different resolutions and the corresponding encoding information are collected to build a large-scale dataset. Finally, the model is embedded into the HEVC coding architecture to predict the result of CU partitioning in advance, which can effectively reduce the coding complexity caused by CU partitioning by eliminating the recursive rate distortion optimization (RDO) calculation process in the original CU partitioning method. Compared with the official HEVC test model (HM16.20), the proposed UcuNet reduces the average coding time by 68.13% while BD-BR is only decreased by 2.63%. [ABSTRACT FROM AUTHOR]

The High-Efficiency Video Coding (HEVC) standard has high compression efficiency. This efficiency is achieved at the expense of increasing the computational complexity. The HEVC encoder has the hierarchical search for optimal Coding Unit (CU) partitioning. It is based on rate-distortion optimization. Various solutions are proposed to reduce the encoding time. But, the machine learning-based methods have more effective in reducing the encoding time. Yet, deep learning tools have a relatively high computational load. So, in this paper a new low complexity convolutional neural network has been designed. It is called Convolutional Neural Network-based CTU Partitioner (CNNCP). It reduces the computational complexity of the HEVC encoding. The CNNCP takes the CTU luminance component and the quantization parameter (QP) as inputs, and provides the CU depth matrix in output at once. The CNNCP does not follow the hierarchical approach. Thus, it has a fixed computation structure that facilitates the use of parallel processing tools. The CNNCP has a simple structure with a least number of parameters, and thus, it has the least computational complexity. It has been trained and tested with a large database for all QP values. The results show that it reduced the encoding time by more than 90%, and makes it suitable for real-time applications. [ABSTRACT FROM AUTHOR]

This research proposes a novel scalable Discrete Cosine transform (DCT) processor. It is based on a shared-resource enhanced Coordinate Rotation Digital Computer (CORDIC) unit, in a modified Loeffler architecture. All micro-rotation operations have moved to the last stages, and are implemented as one unified block in an overlapped form to reduce the utilization area. Also, the beginning stages consist of multiple-delay feedback butterfly units. The hardware-efficient pipelining method of the processor is another reason to reduce the power consumption. This is important while the purpose is resource-constraint IoT devices whereas establishing strong edge servers is not feasible everywhere. Especially in the case of modern video compression standards which demand higher points for DCT such as high efficiency video coding (HEVC) low-power design is crucial. Ubiquitous computing, Big Data and Cloud Services, IoT-Enabled Web are computation-intensive applications. Thus, power reduction of wireless networks of sensors such as, Bluetooth, RFID Wi-Fi, and smartphones, tablets, camcorders are in great demand. In-order input, and output is the proposed design’s advantage. Furthermore, due to utilization of shared-resource CORDIC-II unit, and reduction of adding, shifting operations, in size, and number, the architecture has high performance in short word lengths (WL) in comparison to state-of-the-art DCT processors. [ABSTRACT FROM AUTHOR]

Recently, specifically designed video codecs have been preferred due to the expansion of video data in Internet of Things (IoT) devices. Context Adaptive Binary Arithmetic Coding (CABAC) is the entropy coding module widely used in recent video coding standards such as HEVC/H.265 and VVC/H.266. CABAC is a well known throughput bottleneck due to its strong data dependencies. Because the required context model of the current bin often depends on the results of the previous bin, the context model cannot be prefetched early enough and then results in pipeline stalls. To solve this problem, we propose a prediction-based context model prefetching strategy, effectively eliminating the clock consumption of the contextual model for accessing data in memory. Moreover, we offer multi-result context model update (MCMU) to reduce the critical path delay of context model updates in multi-bin/clock architecture. Furthermore, we apply pre-range update and pre-renormalize techniques to reduce the multiplex BAE's route delay due to the incomplete reliance on the encoding process. Moreover, to further speed up the processing, we propose to process four regular and several bypass bins in parallel with a variable bypass bin incorporation (VBBI) technique. Finally, a quad-loop cache is developed to improve the compatibility of data interactions between the entropy encoder and other video encoder modules. As a result, the pipeline architecture based on the context model prefetching strategy can remove up to 45.66% of the coding time due to stalls of the regular bin, and the parallel architecture can also save 29.25% of the coding time due to model update on average under the condition that the Quantization Parameter (QP) is equal to 22. At the same time, the throughput of our proposed parallel architecture can reach 2191 Mbin/s, which is sufficient to meet the requirements of 8 K Ultra High Definition Television (UHDTV). Additionally, the hardware efficiency (Mbins/s per k gates) of the proposed architecture is higher than that of existing advanced pipeline and parallel architectures. [ABSTRACT FROM AUTHOR]

The implementation of intra-frame prediction algorithm in High Efficiency Video Coding (HEVC) on dedicated hardware cannot meet the requirements of flexible switching between various application scenarios such as HD and mobile video, resulting in poor coding performance and low utilization of hardware resources. To solve this problem, a new implementation method of intra-frame prediction algorithm on the reconfigurable array processor is proposed. The method is based on the state monitoring mechanism. When the idle processing unit is detected, a new execution task is delivered, and the flexible switching between different mapping schemes is realized according to the execution state of the processing unit, so as to achieve the dynamic self-reconstruction of the algorithm execution process. The experimental results show that, compared with the implementation of the intra prediction algorithm on the dedicated processor, the hardware resources are reduced by 33.6% and the number of clock cycles is reduced by 16.2%. Compared with the test results of HM16.7 official software, the average image quality is improved. [ABSTRACT FROM AUTHOR]

Nowadays, High Efficiency Video Coding (HEVC) is widely used in video compression techniques due to the utilization of less bit rate than Advanced Video Coding (AVC) encoders. Motion Estimation (ME) is a vital task in HEVC video compression technique which consumes more encoding time. Various algorithms are proposed in existing studies to deal with ME process in HEVC. Full Search Motion Estimation (FSME) method is most suitable for HEVC because, it is better in terms of high data flow and operating speed. In this work, Particle Swarm Optimization (PSO) is proposed to optimize Sum of Absolute Difference (SAD) calculation value hence, it is named as Enhanced FSME (EFSME). SAD calculation of EFSME not requires adder tree, comparison block and control unit. Hence, there is great reduction in area, power and operating frequency. Proposed work is implemented in Xilinx ZYNQ XA7Z010 FPGA board and it is evaluated by means of power, area and operational frequency. For HEVC, EFSME outputs 65 mW as power consumption rate which is 28.94% lower than Vayalil et al. (2017), 3.03% lesser than Xu et al. (2018) and 80.05% lesser than Singh and Ahamed (2018). Moreover, the proposed design accomplishes 431.906 MHz as operating speed that is 0.44% greater than Vayalil et al. (2017), 20.95% higher than Xu et al. (2018) and 53.35% higher than Singh and Ahamed (2018). Overall, the simulation results proved that the proposed enhanced architecture is better than the existing methodologies in terms of different parameters. [ABSTRACT FROM AUTHOR]

Abstract As High Efficiency Video Coding (HEVC) is a worldwide popular video coding standard, the steganography of HEVC videos has gained more and more attention. Prediction unit (PU) is one of the most important innovative modules of HEVC; thus, PU partition mode-based steganography is becoming a novel branch of HEVC steganography. However, the embedding capacity of this kind of steganography is limited by the types of PU partition modes. To solve the problem, modified exploiting modification direction (EMD)-coded PU partition mode-based steganography is proposed in this paper, which can hide a secret digit in a (2 n + x − 1)-ary notational system in a pair of PU partition modes and thus enlarging the capacity. Furthermore, two mapping patterns for PU partition modes are analyzed, and the one that performs the better is selected as the final mapping pattern. Firstly, 8 × 8- and 16 × 16-sized PU partition modes are recorded according to the optimal mapping pattern in the video encoding process. Then, PU partition modes are modified by using the proposed method to satisfy the requirement of secret information. Finally, the stego video can be obtained by re-encoding the video with the modified PU partition modes. Experimental results show that the embedding capacity can be significantly enlarged, and compared with the state-of-the-art work, the proposed method has much larger capacity while keeping high visual quality.

This work proposes a rate control model based on deep convolutional features to improve the video coding performance of the HEVC encoders under the random access (RA) configuration. The proposed algorithm extracts high-level features from the original and previous coded frames using a pretrained visual geometry group (VGG-16) model by considering characteristics of a different temporal layer for the RA configuration. Subsequently, R– $\lambda $ parameters (alpha and beta), bit allocation, $\lambda $ estimation, and quantization parameter decision at frame-level are formulated by utilizing the extracted high-level features to maintain video quality and bitrate accuracy control. In addition, bit allocation at the group-of-picture (GOP)-level rate control is proposed with perceptual-based thresholding to control smooth bitrates and visual quality between adjacent GOPs. The results verify that the proposed algorithm is efficient in coding performance and bit accuracy by keeping visual quality. Compared with the existing R– $\lambda $ rate model in HM-16.20, the proposed models can achieve an average BD-rate gain of −4.39% and −8.74% in PSNR and MSSSIM metrics for the RA configuration, respectively.

Dynamic point clouds (DPC) are new media storage formats that allow end-users to watch objects/scenes in a three-dimensional (3D) sense. It can be displayed from different angles throughout time. However, the raw size of a point cloud is huge because there can be millions of points (each containing color triplet and location triplet information) in a point cloud, and there can be multiple point clouds in a DPC. Video-based point cloud compression (V-PCC) is developed to project a 3D point cloud to 2D images: attribute, geometry, and occupancy images. After padding, the 2D images are compressed using the well-established high-efficiency video coding (HEVC). In this study, we first employ an occupancy image to propose a blocky occupancy flag (BOF), to denote the occupancy information on “a block basis”. For coding attribute and geometry images, we use a BOF to develop a fast coding unit (CU) algorithm for early termination of the CU search recursion. We also utilize the geometry images to calculate the 2D and 3D information of each pixel, for 2D/3D spatial homogeneity of the pixels to design fast CU decision. In addition, we proposed a modified rate-distortion optimization for different color components considering the picture order count (POC) structure in HEVC/V-PCC. Finally, we propose an HEVC input pixel modification method based on a BOF to reduce the unnecessary information to be coded for attribute images. Compared with the state-of-the-art fast V-PCC encoding method, the proposed work outperforms by up to 2.31% in Bjøntegaard delta bit rates (BDBR) (with very slight loss by only up to 0.38%), and improves the time saving performances by up to 7.84% for two different testing datasets.

Versatile Video Coding (VVC/H.266) is the next-generation international video coding standard and a successor to the widespread High Efficiency Video Coding (HEVC/H.265). This paper analyzes the rate-distortion-complexity characteristics of the VVC reference software (VTM10.0) by using HEVC reference software (HM16.22) as an anchor. In this independent study, the rate-distortion performance of VTM was benchmarked against HM with the objective PSNR, SSIM, and VMAF quality metrics and the associated encoder and decoder complexities were profiled at function level using Intel VTune Profiler on Intel Xeon E5-2699 v4 22-core processors. For a fair comparison, all our experiments were conducted under the VTM common test conditions (CTC) that define 10-bit configurations of the VTM codec for the addressed All Intra (AI), Random Access (RA), and Low Delay B (LB) conditions. The VTM CTC test set was also extended with complementary 4K UHD sequences to elaborate RD characteristics with higher resolutions. According to our evaluations, VTM improves the average coding efficiency over HM, depending on quality metric, by 23.0-23.9% under the AI condition, 33.1-36.6% under the RA condition, and 26.7-29.5% under the LB condition. However, the coding gain of VTM comes with $34.0\times $ , $8.8\times $ , and $7.5\times $ encoding complexity over that of HM under the AI, RA, and LB conditions, respectively. The corresponding overhead of the VTM decoder stays steady at $1.8\times $ across all conditions. This study also pinpoints the most complex parts of the VTM codec and discusses practical implementation aspects of prospective real-time VVC encoders and decoders.

Screen Content (SC) videos require proper tools to handle their special characteristics since they include repeated regions, sharp edges, and limited number of colors. Therefore, SC extension to High Efficiency Video Coding (HEVC) standard has been released for this purpose. SC extension has new tools such as Intra Block Copy (IBC) and Palette (PLT) mode. These tools improve the coding efficiency, but they come with a huge computation complexity. In this paper, we propose a scheme to reduce the encoding time of SC encoder. It has two algorithms based on the Decision Rule (DR) machine learning technique. The first algorithm, which is called Mode Skipping (MS), is used to skip the unnecessary SC mode checking. Early Pruning Termination (EPT), which is the second algorithm, is used to stop the partitioning process early. A small number of features has to be calculated for the trained models. The proposed scheme was implemented then simulated by using the standard software test model SCM-6. The experimental results show that the time complexity is reduced by 37.84% on average while the Bjøntegaard delta bit-rate (BD-R) increases by 1.34% only.

With the popularity of cloud computing and High Efficiency Video Coding (HEVC) codec, commutative encryption and data hiding schemes designed for HEVC has attracted great attention. This paper proposes an improved commutative encryption and data hiding scheme for HEVC video. In order to produce sufficient visual distortion, in addition to the syntax elements of context-based adaptive binary arithmetic coding in bypass mode, some syntax elements in regular mode including luma intra-prediction mode (IPM) and chroma IPM are selected for encryption. Quantized transform coefficient (QTC)'s amplitude is exploited for data embedding. An improved data embedding technology is designed by combining bin-string pairing, which improves the embedding capacity to a certain extent. In the framework, video encryption and data hiding are fully commutative, and the marked encrypted bitstream is format compatible. Experimental results demonstrate that the improved scheme can enhance the security while increasing the embedding capacity. [ABSTRACT FROM AUTHOR]

Versatile Video Coding (VVC/H.266) is an emerging successor to the widespread High Efficiency Video Coding (HEVC/H.265) and is shown to double the coding efficiency for the same subjective visual quality. Nevertheless, VVC still adopts the similar hybrid video coding scheme as HEVC and thereby sets the scene for reusing many HEVC coding tools and techniques as is or with minor modifications. This paper explores the feasibility of developing a practical software VVC intra encoder from our open-source Kvazaar HEVC encoder. The outcome of this work is called uvg266 VVC intra encoder that is distributed under the same permissive 3-clause BSD license as Kvazaar. uvg266 inherits the optimized coding flow of Kvazaar and all upgradable Kvazaar intra coding tools, but it also introduces basic VVC intra coding tools not available in HEVC. To the best of our knowledge, this is the first work to describe the implementation details of upgrading an HEVC encoder to a VVC encoder. The rapid development time with promising coding performance make our proposal a viable approach over the encoder development from scratch. [ABSTRACT FROM AUTHOR]

Continuous growth of video traffic and video services, especially in the field of high resolution and high-quality video content, places heavy demands on video coding and its implementations. High Efficiency Video Coding (HEVC) standard doubles the compression efficiency of its predecessor H.264/AVC at the cost of high computational complexity. To address those computing issues high-performance video processing takes advantage of heterogeneous multiprocessor platforms. In this paper, we present a highly performance-optimized HEVC transform and quantization kernel with all-zero-block (AZB) identification designed for execution on a Graphics Processor Unit (GPU). Performance optimization strategy involved all three aspects of parallel design, exposing as much of the application’s intrinsic parallelism as possible, exploitation of high throughput memory and efficient instruction usage. It combines efficient mapping of transform blocks to thread-blocks and efficient vectorized access patterns to shared memory for all transform sizes supported in the standard. Two different GPUs of the same architecture were used to evaluate proposed implementation. Achieved processing times are 6.03 and 23.94 ms for DCI 4K and 8K Full Format, respectively. Speedup factors compared to CPU, cuBLAS and AVX2 implementations are up to 80, 19 and 4 times respectively. Proposed implementation outperforms previous work 1.22 times.

In this manuscript, new quadrant-based search algorithm with zero motion prejudgment is proposed for motion estimation (ME) in HEVC (High Efficiency Video Coding) standard. The HEVC standard is used to obtain efficient output with low motion estimation time. The proposed quadrant-based search algorithm is a fast block matching algorithm that obtain better block matching amid the current block and reference block. The zero motion prejudgment (ZMP) method is used to find the block, whether it is motion or static and it is used for decreasing the computational complexity (CC) in the proposed quadrant-based search algorithm. The proposed quadrant-based search algorithm with ZMP technique for motion estimation in HEVC is implemented on the FPGA hardware platform. The entire architecture is executed in Verilog HDL with Virtex-5 technology and integrated with Xilinx ISE Design Suite 14.5. The results are integrated into the CIF (352 × 288 pixels) and HD (1280 × 720 pixels) video input sequence. The evaluation metrics like PSNR, Motion estimation time, sum of absolute difference (SAD) value are analyzed with existing method like hexagon, adaptive root pattern algorithm, and diamond search algorithm. Then the hardware parameters like power consumption and maximum operating frequency are measured. The hardware utilization is reduced and the power consumption of the proposed model is diminished to 0.143 W. The maximal operating frequency of the proposed model is 440.470 MHz. The experimental outcomes demonstrate that the proposed motion evaluation approach in HEVC is more effective than existing algorithms. [ABSTRACT FROM AUTHOR]

Video coding removes spatial, temporal, and statistic redundancies. After H.265, to further improve the coding efficiency, many efforts have been dedicated to removing the perceptual redundancy by using human perception-based methods. Just noticeable difference (JND) gives a good approximation for the human visual system and provides a valuable solution to remove the perceptual redundancy for perceptual video coding (PVC). However, there are still problems in the PVC architecture and the JND profile. One is although the whole discrete cosine transform (DCT) block are suppressed, there are still many transform coefficients below the suppression levels which are not adequately suppressed. Another problem is, to the best of our knowledge, most JND profiles are measured by image-based test methods and past display equipment. However, compared to images, videos exhibits temporal characteristics, and the current trend of the display equipment is towards full high definition. To solve these problems, we first propose a high efficiency video coding (HEVC)-compliant PVC architecture, where the coefficients in a DCT block can be adequately suppressed in a whole block manner. Second, we propose a video-based test method to model the temporal masking (TM) effect, called TM-JND. Experimental results show that the proposed TM-JND model can more accurately estimate the JND values for today's display equipment and videos, avoiding the overestimate of the JND values like other existing models. The proposed PVC architecture achieves a significant bitrate reduction with a negligible subjective quality loss, compared with the HEVC test model HM 16.9. [ABSTRACT FROM AUTHOR]

The Digital Video Broadcasting (DVB) has proposed to introduce the Ultra-High Definition services in three phases: UHD-1 phase 1, UHD-1 phase 2 and UHD-2. The UHD-1 phase 2 specification includes several new features such as High Dynamic Range (HDR) and High Frame-Rate (HFR). It has been shown in several studies that HFR (+100 fps) enhances the perceptual quality and that this quality enhancement is content-dependent. On the other hand, HFR brings several challenges to the transmission chain including codec complexity increase and bit-rate overhead, which may delay or even prevent its deployment in the broadcast echo-system. In this paper, we propose a Variable Frame Rate (VFR) solution to determine the minimum (critical) frame-rate that preserves the perceived video quality of HFR video. The frame-rate determination is modeled as a 3-class classification problem which consists in dynamically and locally selecting one frame-rate among three: 30, 60 and 120 frames per second. Two random forests classifiers are trained with a ground truth carefully built by experts for this purpose. The subjective results conducted on ten HFR video contents, not included in the training set, clearly show the efficiency of the proposed solution enabling to locally determine the lowest possible frame-rate while preserving the quality of the HFR content. Moreover, our VFR solution enables significant bit-rate savings and complexity reductions at both encoder and decoder sides. [ABSTRACT FROM AUTHOR]

In this study, an efficient scheme for hiding data directly in partially encrypted versions of high efficiency video coding (HEVC) videos is proposed. The content owner uses stream cipher to selectively encrypt some HEVC‐CABAC bin strings in a format‐compliant manner. Then, the data hider embeds the secret message into the encrypted HEVC videos using the specific coefficient modification technique. Consequently, it can be used in third‐party computing environments (more generally, cloud computing). For security and privacy purposes, service providers cannot access the visual content of the host video. As the coefficient is only slightly modified, the quality of the decrypted video is satisfactory. The encrypted and marked bitstreams meet the requirements of format compatibility, and have the same bit rate. At the receiving end, data extraction can be performed in the encrypted domain or decrypted domain that can be adapted to different application scenarios. Several standard video sequences with different resolutions and contents have been used for experimental evaluation.

In this paper, an improved commutative encryption and data hiding scheme for bit rate preservation of High Efficiency Video Coding (HEVC) is proposed. To achieve commutative property, one set of syntax elements within the HEVC standard is utilized for data hiding, while another set is exploited for encryption. Specifically, the syntax elements including the sign of quantized transform coefficient (QTC), the sign of motion vector difference (MVD), and the magnitude of MVD are encrypted to perceptually distort the video. On the other hand, an improved QTC modification method is employed for data hiding to increase the embedding capacity. The proposed framework produces the format compliant bitstream and allows the hidden information to be extracted regardless of the video being in the encrypted or decrypted from. Experimental results demonstrate that the proposed scheme keeps the bit rate unchanged while increasing the embedding capacity.

The hierarchical quadtree partitioning of Coding Tree Units (CTU) is one of the striking features in HEVC that contributes towards its superior coding performance over its predecessors. However, the brute force evaluation of the quadtree hierarchy using the Rate-Distortion (RD) optimisation, to determine the best partitioning structure for a given content, makes it one of the most time-consuming operations in HEVC encoding. In this context, this paper proposes an intelligent fast Coding Unit (CU) size selection algorithm to expedite the encoding process of HEVC inter-prediction. The proposed algorithm introduces (i) two CU split likelihood modelling and classification approaches using Support Vector Machines (SVM) and Bayesian probabilistic models, and (ii) a fast CU selection algorithm that makes use of both offline trained SVMs and online trained Bayesian probabilistic models. Finally, (iii) a computational complexity to coding efficiency trade-off mechanism is introduced to flexibly control the algorithm to suit different encoding requirements. The experimental results of the proposed algorithm demonstrate an average encoding time reduction performance of 53.46%, 61.15%, and 58.15% for Low Delay B, Random Access, and Low Delay P configurations, respectively, with Bjøntegaard Delta-Bit Rate (BD-BR) losses of 2.35%, 2.9%, and 2.35%, respectively, when evaluated across a wide range of content types and quality levels.

In this paper, we propose a perceptual adaptive quantization based on a deep neural network on high efficiency video coding (HEVC) for bitrate reduction while maintaining subjective visual quality. The proposed algorithm adaptively determines frame-level QP values for different picture types of the hierarchical coding structure in HEVC by taking into account the high-level features extracted from the original and previously reconstructed pictures. A predefined model based on the visual geometry group (VGG-16) network is exploited to extract the high-level features for subjective visual characteristics. Furthermore, the Lagrange multiplier for each frame is also adaptively determined by involving the proposed features for deciding the appropriate parameter of the Lagrange multiplier that can be used for rate-distortion optimization during the encoding process. Experimental results reveal that the proposed perceptual adaptive QP selection can facilitate bitrate savings up to 65.73% and 47.68% and improve the BD-rate based on SSIM by approximately 20.68% and 14.27% under low-delay-P and random-access coding structures, respectively, with very minimal visual quality degradation when compared to HM-16.20 without adaptive QP selection.

This paper proposes an optimal rate control model based on deep neural network (DNN) features to improve the coding tree unit (CTU)-level rate control in high-efficiency video coding for conversational videos. The proposed algorithm extracts high-level features from the original and previously reconstructed CTU blocks based on a predefined DNN model of the visual geometry group (VGG-16) network. Then, the correlation of the high-level feature and quantization parameter (QP) values of previously coded CTUs is explored for subjective visual characteristics to estimate the CTU-level rate control model parameters (alpha and beta) and the bit allocation of each CTU. Therefore, this paper also proposes a new model for Lambda estimation for each CTU by improving its relationship with the estimated bits per pixel to control the rate and relative distortion. Furthermore, the Lambda and QP boundary settings were adjusted based on the proposed perceptual model to ensure the rate control accuracy of each CTU. The results of experiments with the proposed algorithm, when compared to the rate control model in HM-16.20, reveal higher bitrate accuracy and an average BD-rate gain based on PSNR, SSIM, and MSSSIM metrics using the low-delay-P configuration.

Copyright of Frontiers of Information Technology & Electronic Engineering is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In the last 17 years, since the finalization of the first version of the now-dominant H.264/Moving Picture Experts Group-4 (MPEG-4) Advanced Video Coding (AVC) standard in 2003, two major new generations of video coding standards have been developed. These include the standards known as High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC). HEVC was finalized in 2013, repeating the ten-year cycle time set by its predecessor and providing about 50% bit-rate reduction over AVC. The cycle was shortened by three years for the VVC project, which was finalized in July 2020, yet again achieving about a 50% bit-rate reduction over its predecessor (HEVC). This article summarizes these developments in video coding standardization after AVC. It especially focuses on providing an overview of the first version of VVC, including comparisons against HEVC. Besides further advances in hybrid video compression, as in previous development cycles, the broad versatility of the application domain that is highlighted in the title of VVC is explained. Included in VVC is the support for a wide range of applications beyond the typical standard- and high-definition camera-captured content codings, including features to support computer-generated/screen content, high dynamic range content, multilayer and multiview coding, and support for immersive media such as 360° video. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Harbin Institute of Technology. Social Sciences Edition / Haerbin Gongye Daxue Xuebao. Shehui Kexue Ban is the property of Harbin Institute of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

High efficiency video coding is the most widely used video coding standard. It has higher coding performance compared with its predecessor, H.264, but it also has higher computational complexity. Interprediction is the most computationally intensive part of the entire video encoding process. Selecting the optimal interprediction mode by the rate–distortion cost calculation function requires substantial complex calculation and memory access, thus greatly increasing the difficulty of real-time hardware encoding. This study proposes to replace the traditional complex error square sum calculation with an estimation method for distortion and rate. The estimation of distortion uses the Hadamard-transformed sum of absolute transformation difference instead of the complex calculation of the sum of squared difference, whereas the estimation of rate is obtained by weighting the number of prediction units (PUs). The experiment proves that the proposed interprediction rate–distortion cost calculation model can greatly reduce computational complexity when BD-rate is increased by 3.02%. In hardware implementation, the value of rate can be obtained by indexing the number of PUs, and the resource expenditure is small. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) is the latest video coding standard that supports high resolution videos by providing approximately twice the compression efficiency as compared to its previous standard H.264. Motion Estimation (ME) in HEVC is the most computation-intensive block as a result it becomes a bottleneck in the design of the encoder while implementing video applications on various computing platforms such as general purpose and embedded processors. So developing computational efficient architectures on Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC) platforms is inevitable. This paper proposes a fast hybrid search pattern algorithm and its hardware architecture for encoding UHD videos. The proposed Integer ME (IME) algorithm requires an average of 11.19% less encoding time than the default Test Zone Search (TZS) algorithm in HM reference software with compromising decrement in PSNR and increment in bit rate. The proposed architecture is implemented in both FPGA and ASIC platform with TSMC 90 nm technology library. It consumed 32-33% of resources in Virtex-7 FPGA and 2784.4 K equivalent gate count (in terms of NAND) and 18 kB of memory, respectively. The results show that maximum frequency of the proposed architecture is 162 MHz and a total power consumption is 463.4 mW. The architecture provides a maximum throughput of 2.78 Gpixels/sec due to it process 32 × 32 CU comparatively much less clock cycles (59.5) as compared to the state-of-the-art literature. Further, it supports 8K UHD (8192 × 4320) @ 78 fps. [ABSTRACT FROM AUTHOR]

The introduction of the new coding tools in HEVC has brought significant bitrate savings compared to the previous standard, H.264/AVC. In both standards, the mean square error (MSE) is used for measuring distortion in the rate distortion optimization process of the coding unit structure and mode selection. However, MSE is not a good measure to use for measuring visual quality as it poorly correlates with human perception. Integration of a video quality metric based on the characteristics of the Human Visual System (HVS) inside the rate distortion optimization procedure is expected to improve the compression efficiency of video coding. In this paper, the PSNR-HVS measure is used in the rate distortion optimization process for the coding unit structure and mode selection. In the first step, we find the scaling factor for the Lagrangian multiplier based on the proposed perceptual approach. In the second step, we find optimal Lagrangian multiplier depending on the quantization parameter. The compression efficiency of the proposed approach is compared to that of HEVC. Simulations prove that the proposed approach yields higher compression efficiency. [ABSTRACT FROM AUTHOR]

In high efficiency video coding (HEVC), transformation and quantization are separately performed to eliminate the perceptual redundancy of visual signals. However, a uniform quantizer can inevitably degrade the compression efficiency of fixed transform matrices due to varying space-frequency characteristics of video content. This paper introduces a joint optimization of transform and quantization approach for video coding. First, we compute a content dependent transform from the reconstructed reference by a fast Karhunen-Loéve transform (KLT). Second, using a template-based rate regularization, we jointly optimize transform and quantization (JOTQ) as a rate constrained optimization problem and obtain a feasible solution to improve coding performance. Finally, we design fast algorithms and early terminations to reduce the computational complexity of JOTQ. The experimental results show that JOTQ outperforms several previous methods by providing Bjøntegaard Delta rate reductions of 4.11% and 3.38% on average under the low-delay and random-access configuration, respectively.