Search

Recently, machine learning-based image compression has attracted increasing interests and is approaching the state-of-the-art compression ratio. But unlike traditional codec, it lacks a universal optimization method to seek efficient representation for different images. In this paper, we develop a plug-and-play optimization framework for seeking higher compression ratio, which can be flexibly applied to existing and potential future compression networks. To make the latent representation more efficient, we propose a novel latent optimization algorithm to adaptively remove the redundancy for each image. Additionally, inspired by the potential of side information for traditional codecs, we introduce side information into our framework, and integrate side information optimization with latent optimization to further enhance the compression ratio. In particular, with the joint side information and latent optimization, we can achieve fine rate control using only single model instead of training different models for different rate-distortion trade-offs, which significantly reduces the training and storage cost to support multiple bit rates. Experimental results demonstrate that our proposed framework can remarkably boost the machine learning-based compression ratio, achieving more than 10% additional bit rate saving on three different representative network structures. With the proposed optimization framework, we can achieve 7.6% bit rate saving against the latest traditional coding standard VVC on Kodak dataset, yielding the state-of-the-art compression ratio. [ABSTRACT FROM AUTHOR]

A tunable active-noise-control (ANC) circuit topology for headsets used in different applications is presented in this article. In the current consumer headset market, ANC is a mature technology that is commonly applied to wireless headsets connected to smartphones for listening to music and making phone calls. The development of ANC headsets has resulted in low-cost and simple devices due to the built-in ANC registers in the digital circuit. Digital circuit implementation is important to provide ultra-low latency processing for different algorithms compared with using digital signal processors (DSPs). However, a limitation associated with these built-in ANC filters is that the ANC digital circuit has been designed only for the wireless headset consumer market. Consequently, ANC headsets can only be used for designated features. ANC tuned for planes by eliminating low-frequency engine noise has been commonly used in the past twenty years. Pass-through (PT) amplification is also commonly tuned to allow users to hear external sounds and have conversations without having to remove their headset. In addition, ANC and PT amplification are also not allowed to operate simultaneously on the existing chipsets. In this paper, an ANC circuit topology with ultra-low latency processing is presented, where the ANC response of the headset can be tuned without any restriction imposed by the built-in ANC filters. The proposed ANC circuit topology consists of a commercial ANC chipset with an external audio codec to provide tunable characteristics. It is achieved by fixed maximum noise cancellation of ANC chipset being modified by summing with the signal from audio-codec path via an audio mixer. Acoustic performance for several headset applications implemented using the proposed circuit is validated. This simple circuit topology can be easily developed and adapted to the current headset consumer market for consumer-selective ANC in different environments. [ABSTRACT FROM AUTHOR]

3D point cloud data, as an immersive detailed data source, has been increasingly used in numerous applications. To deal with the computational and storage challenges of this data, it needs to be compressed before transmission, storage, and processing, especially in real-time systems. Instead of decoding the compressed data stream and subsequently conducting downstream tasks on the decompressed data, analyzing point clouds directly in their compressed domain has attracted great interest. In this paper, we dive into the realm of compressed point cloud classification (CPCC), aiming to achieve high point cloud classification accuracy in a bitrate-saving way by ensuring the bit stream contains a high degree of representative information of the point cloud. Edge information is one of the most important and representative attributes of the point cloud because it can display the outlines or main shapes. However, extracting edge points or information from point cloud models is challenging due to their irregularity and sparsity. To address this challenge, we adopt an advanced edge-sampling method that enhances existing state-of-the-art (SOTA) point cloud edge-sampling techniques based on attention mechanisms and consequently develop a novel CPCC method "CPCC-PES" that focuses on point cloud's edge information. The result obtained on the benchmark ModelNet40 dataset shows that our model has superior rate-accuracy trade-off performance than SOTA works. Specifically, our method achieves over 90% Top-1 Accuracy with a mere 0.08 bits-per-point (bpp), marking a remarkable over 96% reduction in BD-bitrate compared with specialized codecs. This means that our method only consumes 20% of the bitrate of other SOTA works while maintaining comparable accuracy. Furthermore, we propose a new evaluation metric named BD-Top-1 Accuracy to evaluate the trade-off performance between bitrate and Top-1 Accuracy for future CPCC research. [ABSTRACT FROM AUTHOR]

This paper presents an evaluation of the latest MPEG-5 Part 2 Low Complexity Enhancement Video Coding (LCEVC) for video streaming applications using best effort protocols. LCEVC is a new video standard by MPEG, which enhances any base codec through an additional low bitrate stream, improving both video compression efficiency and and transmission. However, there is an interplay between packetization, packet loss visibility, choice of codec and video quality, which implies that prior studies with other codecs may be not as relevant. The contributions of this paper is, therefore in twofold: It evaluates the compression performance of LCEVC and then the impact of packet loss on its video quality when compared to H.264 and HEVC.The results from this evaluation suggest that, regarding compression, LCEVC outperformed its base codecs, overall in terms average encoding bitrate savings when using the constant rate factor (CRF) rate control. For example at a CRF of 19, the average encoding bitrate was reduced by 18.7% and 15.8% when compared with the base H.264 and HEVC codecs respectively. Furthermore, LCEVC produced better visual quality across the packet loss range compared to its base codecs and the quality only started to decrease once packet loss exceeded 0.8-1%, and decreases at a slower pace compared to its equivalent base codecs. This suggests that the LCEVC enhancement layer also provides error concealment. The results presented in this paper will be of interest to those considering the LCEVC standard and expected video quality in error-prone environments [ABSTRACT FROM AUTHOR]

Over the years, there has been growing interest in multimedia services, especially in the video domain, where firms and subscribers require higher resolutions, framerates, and sampling precision. This results in a huge amount of data that needs to be processed, stored, and transmitted. As a result, researchers face the challenge of developing new compression standards that can reduce the amount of data while maintaining the same quality. In this paper, the compression performance of the latest and most commonly used video codecs, namely H.266/VVC, AV1, H265/HEVC, and H.264/AVC was examined. The test set included seven sequences of various content at 8K, Ultra HD (UHD), and Full HD (FHD) resolutions, encoded to bitrates ranging from 1 to 15 Mbps for FHD and UHD resolutions and from 5 to 50 Mbps for 8K resolution. Objective quality metrics, such as peak signal-to-noise ratio (PSNR), the structural similarity index (SSIM), and video multi-method assessment fusion (VMAF) were used to measure codec performance. The results showed that H.266/VVC outperformed all other codecs, namely H.264/AVC, H.265/HEVC, and AV1, in terms of the Bjøntegaard delta (BD) model. The average bitrate savings were approximately 78% for H.266/VVC, 63% for AV1, and 53% for H.265/HEVC relative to H.264/AVC, 59% for H.266/VVC and 22% for AV1 compared to H.264/AVC, and 46% for H.266/VVC relative to AV1 (all for 8K resolution). The results also showed that codec performance varied depending on resolution, with higher resolutions showing greater efficiency for newly developed codecs, such as H.266/VVC and AV1. This confirms the fact that the H.266/VVC and AV1 codecs were primarily developed for videos at high resolutions, such as 8K and/or UHD. [ABSTRACT FROM AUTHOR]

It is challenging to enable fast channel switching and packet-loss repair in low-delay live video distribution without negatively influencing the steady-state viewing performance. For example, regularly breaking the inter-frame dependency by introducing intra-predicted keyframes enables random access, but is costly in terms of rate-distortion performance. For this reason, the keyframe-injection method minimizes the impact by sending a compression-efficient normal video stream to all end-users. As accompaniment, a companion stream that solely consists of keyframes is sporadically used for only those users that switch channels or experience packet loss. This paper describes the requirements to implement keyframe injection in three video coding standard generations (H.264/AVC, H.265/HEVC, and H.266/VVC). We evaluated the impact that keyframe injection has on the quality of the video in terms of a decrease in VMAF, PSNR and SSIM. We demonstrate that the quality reduction caused by keyframe insertion is generally low, meaning that keyframe injection typically is imperceptible. However, drift-error artifacts become perceptible over time for rare outliers. Moreover, we pinpointed the cause of this worst-case artifact type to be halfpel interpolation. As a solution, codecs can disable subpel motion estimation, and future standards could design their filters more carefully. Lastly, it should be noted that keyframe injection will only be applied sporadically when users require a random access or experience packet loss, and only to those users. Most interestingly, all other users receive a compression-efficient stream wherein the inter-frame dependency is not artificially broken at regular short intervals. As such, our proposed solution makes low-latency video distribution efficient and viable in multiple coding standards. [ABSTRACT FROM AUTHOR]

• Conditionally and directly skip all non-first pre-coding. • In I picture, for a CU with top-left location (x, y), if both x and y are multiples of 64, then each non-first pre-coding is skipped. • In I picture, for a CU with top-left location (x, y), if at least one of x and y is not multiples of 64 and a set of conditions is satisfied by a non-first pre-coding, then the non-first pre-coding is skipped. • In non-I picture, for CUs at any location, if a set of conditions is satisfied by a non-first pre-coding, then the non-first pre-coding is skipped. • Direct Non-First Pre-coding Skip (DNFPS) significantly reduce the coding complexity with negligible impact on coding efficiency. The third generation of Audio Video Coding Standard (AVS3) adopts sophisticated block partitioning structure with a multi-type tree, which includes Quad-Tree (QT), Binary-Tree (BT), and Extend Quad-Tree (EQT). During multi-type tree recursive block partitioning and pre-coding process of encoding, each Coding Unit (CU) of each Size at each Location (CUeSeL) of the picture is repeatedly pre-coded many times when visiting each branch of each tree. Although there are early termination mechanism, the number of visits for each CUeSeL is still very high and up to more than fifty. This process brings very high coding complexity. In order to reduce the coding complexity of AVS3, this paper proposes a method named Direct Non-First Pre-coding Skip (DNFPS) for a CUeSeL to directly and completely skip all non-first pre-coding under specific conditions. The conditions include, but are not limited to, those related to the first pre-coding. The contributions of the paper are as follows. In I pictures, for a CUeSeL with top-left location (x, y), if both x and y are multiples of 64 or if a set of conditions related to first pre-coding is satisfied, then each non-first pre-coding is skipped, and the coding result of the first pre-coding is reused for each non-first pre-coding skipped. In non-I pictures, each non-first pre-coding is skipped, and the coding result of the first pre-coding is reused for each non-first pre-coding skipped, provided a set of conditions related to first pre-coding is satisfied. The experimental results demonstrate that DNFPS has a negligible impact on coding efficiency under All Intra (AI) configurations, and coding runtime is reduced by 16 %. [ABSTRACT FROM AUTHOR]

• An adaptive HEVC video steganography algorithm is designed, which can effectively resist the PU shift-based steganalysis. • The introduction of multivariate STC allows for smaller modifications with the same embedding capacity. • The superiority of this algorithm over the state-of-the-art video steganography algorithm is verified through extensive experiments. Calibration-based steganalysis methods are a fatal threat to current HEVC steganography algorithms. The current steganography algorithms based on prediction unit (PU) still lack resistance to this type of steganalysis. The reason is that the improvements in HEVC codec provide more potential steganographic space as well as introduce more detectable distortion. Therefore, a HEVC video steganography method that has resistance to PU shift-based steganalysis is proposed in this paper. First, the PU shift phenomenon that exists in PU-based steganography is introduced. Then the existing calibration-based steganalysis method is enhanced. According to the encoding rules of HEVC, the distortion cost function according to the difference of rate distortion cost (RD cost) during recompression is designed, which aims to obtain better resistance to calibration-based steganalysis. Experimental results demonstrate that the steganography algorithm designed in this paper provides excellent resistance ability for calibration-based steganalysis. [ABSTRACT FROM AUTHOR]

Compression method for GPU streaming of discrete light fields is proposed in this paper. Views on the scene are encoded with video codec to enable streaming in real time. Instead of using a classic scheme, all frames are encoded according to one reference frame. Any frame is decoded directly, in a random-access manner that is suitable for light-field rendering methods, where only few frames are necessary on the GPU. The proposed scheme reaches the best decoding quality/time ratio in comparison to other schemes, where all preceding frames need to be decompressed, and all-key-frame video that supports random access, but is extremely large. The proposed method solves the space-requirements and streaming-bandwidth issues using the GPU accelerated decoding, and enables incorporating light-field assets in real-time 3D simulations. Compared to existing methods, the proposal is easy to implement, does not depend on specific video format or extension and is efficient on consumer GPUs. [Display omitted] • Real-time streaming method for light field rendering. • Solves light field issues such as excessive memory usage and slow streaming. • GPU-accelerated approach using nowadays actively developed HW decoders. • Ready to be used with upcoming video codecs. • Better than existing methods in speed, quality, implementation, native support. [ABSTRACT FROM AUTHOR]

In the multimedia era, exploding image and video data highlight the importance of video compression for storage and transmission. The All-Intra structure is a coding mode in HEVC and VVC, in which each frame is encoded using intra coding, and in this paper learned All-Intra coding is explored on the basis of the research of the learned image compression. A channel-wise contextual module based on channel segmentation is introduced to fully exploit non-local information. Then, two distinct attention mechanisms are designed for different feature layers to enhance the effectiveness of the transform network. Additionally, a post-processing module is employed to enhance the quality of decoded frames. Experimental results on the Kodak and Tecnick datasets demonstrate that the proposed method performs better than the majority of the recent learning-based methods and traditional image codecs (BPG, JPEG2000 and JPEG), and also perform better than traditional video codecs in terms of PSNR. • A novel video compression framework specially designed for I frame is introduced. • A channel-wise contextual module enhances context-based entropy modeling. • A post-processing module aims to enhance the quality of the reconstructed frames. • Two distinct attention mechanisms are designed for different feature layers. [ABSTRACT FROM AUTHOR]

The growing demand for high-quality video transmission over bandwidth-constrained networks and the increasing availability of video content have led to the need for efficient storage and distribution of large video files. To improve the latter, this article offers a comparison of six video compression methods without loss of quality. Particularly, H.255, VP9, AV1, convolutional neural network (CNN), recurrent neural network (RNN), and deep autoencoder (DAE). The proposed decision is to use a dataset of high-quality videos to implement and compare the performance of classical compression algorithms and algorithms based on machine learning. Evaluations of the compression efficiency and the quality of the received images were made on the basis of two metrics: PSNR and SSIM. This comparison revealed the strengths and weaknesses of each approach and provided insights into how machine learning algorithms can be optimized in future research. In general, it contributed to the development of more efficient and effective video compression algorithms that can be useful for a wide range of applications. [ABSTRACT FROM AUTHOR]

This letter proposes an augmented scheme of LT codes to improve the decoding success rate. The method involves substituting the 1‐s in full‐degree columns of the generator matrix of conventional LT codes with binary extension field elements and simultaneously increasing the ratio of full‐degree columns in the ideal soliton distribution. For non‐full‐degree columns, we retain the nonzero elements as 1‐s in the conventional LT codes generator matrix to preserve computational efficiency. Compared to conventional LT codes, the proposed method enhances the linear independence of the generator matrix, leading to a higher decoding success rate with minimal data packets. Experimental results demonstrate the effectiveness of the method at improving the performance of LT codes, with close to 100% decoding success rate achieved with around 5% data redundancy. [ABSTRACT FROM AUTHOR]

With the developing explosion in video data delivery, perceptual video coding (PVC) plays an increasingly significant role in video compression. The just noticeable distortion (JND) reflects the tolerance limit of human visual system (HVS) to coding distortion directly, resulting in that JND-based PVC is the most important branch of video coding. This paper provides an extensive overview of JND estimation and JND-based PVC, so as to make interested readers aware of the status quo. The main contribution of this article can be briefly outlined as follows. Firstly, the general description of JND concepts and most existing computation models for JND are to be reviewed systematically. Secondly, most related works about JND-based perceptual image and video coding schemes are introduced, including JND-based coding preprocessing and JND-based codec embedding. Thirdly, in addition to a thorough summary of JND estimation and JND-based PVC, possible future directions and opportunities are analyzed and discussed. • Firstly, the general description of JND concepts and most existing computation models for JND are to be reviewed systematically. • Secondly, most related works about JND-based perceptual image or video coding schemes are introduced, including JND-based coding preprocessing and JND-based codec embedding. • Thirdly, in addition to a thorough summary of JND estimation and JND-based PVC, possible future directions and opportunities are analyzed and discussed. [ABSTRACT FROM AUTHOR]

Voice over internet protocol (VoIP) calls are increasingly transported over computer-based networking due to several factors, such as low call rates. However, point-to-point (P-P) calls, as a division of VoIP, are encountering a capacity utilization issue. The main reason for that is the giant packet header, especially when compared to the runt P-P calls packet payload. Therefore, this research article introduced a method to solve the liability of the giant packet header of the P-P calls. The introduced method is named voice segment compaction (VSC). The VSC method employs the unneeded P-P calls packet header elements to carry the voice packet payload. This, in turn, reduces the size of the voice payload and improves network capacity utilization. The preliminary results demonstrated the importance of the introduced VSC method, while network capacity improved by up to 38.33%. [ABSTRACT FROM AUTHOR]