Search

Advancements in artificial intelligence have significantly increased the use of images and videos in machine analysis algorithms, predominantly neural networks. However, the traditional methods of compressing, storing and transmitting media have been optimized for human viewers rather than machines. Current research in coding images and videos for machine analysis has evolved in two distinct paths. The first is characterized by End-to-End (E2E) learned codes, which show promising results in image coding but have yet to match the performance of leading Conventional Video Codecs (CVC) and suffer from a lack of interoperability. The second path optimizes CVC, such as the Versatile Video Coding (VVC) standard, for machine-oriented reconstruction. Although CVC-based approaches enjoy widespread hardware and software compatibility and interoperability, they often fall short in machine task performance, especially at lower bitrates. This paper proposes a novel hybrid codec for machines named NN-VVC, which combines the advantages of an E2E-learned image codec and a CVC to achieve high performance in both image and video coding for machines. Our experiments show that the proposed system achieved up to − 43.20% and − 26.8% Bjøntegaard Delta rate reduction over VVC for image and video data, respectively, when evaluated on multiple different datasets and machine vision tasks according to the common test conditions designed by the VCM study group in MPEG standardization activities. Furthermore, to improve reconstruction quality, we introduce a human-focused branch into our codec, enhancing the visual appeal of reconstructions intended for human supervision of the machine-oriented main branch. [ABSTRACT FROM AUTHOR]

Analysis of computational complexity is presented for an H.26L video decoder, based on extensive experiments on a general-purpose processor. In addition, platform-independent techniques to optimize an H.26L decoder implementation are given. Comparisons are carried out between our highly optimized version of H.26L, the public reference implementation of H.26L, and a highly optimized H.263+ implementation. Both QCIF and CIF-sized image sequences are used. The results show that with equal visual quality, the bit-rate savings range from 28% to 58%, while the frame decoding speed of H.26L is about 11% better than that of a highly optimized H.263+. Index Terms--H.263, H.263+, H.26L, video coding.

In this paper, we study and analyze the computational complexity of a software-based H.264/AVC baseline profile decoder. Our analysis is based on determining the number of basic computational operations required by a decoder to perform the key decoding subfunctions. The frequency of use of each of the required decoding subfunctions is empirically derived using bitstreams generated from two different encoders for a variety of content, resolutions and bit rates. Using the measured frequencies, estimates of the decoder time complexity for various hardware platforms can be determined. A detailed example is provided to assist readers in deriving their own time complexity estimates. We compare the resulting estimates to numbers measured for an optimized decoder on the Pentium 3 hardware platform. We then use those numbers to evaluate the dependence of the time complexity of each of the major decoder subfunctions on encoder characteristics, content, resolution and bit rate. Finally, we compare an H.264/AVC-compliant baseline decoder to a decoder that is compliant with the H.263 standard, which is currently dominant in interactive video applications. Both 'C' only decoder implementations were compared on a Pentium 3 hardware platform. Our results indicate that an H.264/AVC baseline decoder is approximately 2.5 times more time complex than an H.263 baseline decoder. Index Terms--Complexity, decoder, H.264/AVC, H.26L, JVT.

This paper presents an overview of the transform and quantization designs in H.264. Unlike the popular 8 x 8 discrete cosine transform used in previous standards, the 4 x 4 transforms in H.264 can be computed exactly in integer arithmetic, thus avoiding inverse transform mismatch problems. The new transforms can also be computed without multiplications, just additions and shifts, in 16-bit arithmetic, thus minimizing computational complexity, especially for low-end processors. By using short tables, the new quantization formulas use multiplications but avoid divisions. Index Terms--Integer transforms, periodic quantization, quantization, transforms, video coding, video standards.

Multiple description coding offers interesting solutions for error resilient multimedia communications as well as for distributed streaming applications. In this Letter, we propose a scheme based on H.264/AVC for encoding of image sequences into multiple descriptions. The pictures are split into multiple coding threads. Redundant pictures are inserted periodically in order to increase the resilience to loss and to reduce the error propagation. They are produced with different reference frames than the corresponding primary pictures. We show, given the channel conditions, how to optimally allocate the rates to primary and redundant pictures, such that the total distortion at the receiver is minimized. Extensive experiments demonstrate that the proposed scheme outperforms baseline solutions based on loss and content-adaptive intra coding. Finally, we show how to further reduce the distortion by efficient combination of primary and redundant pictures, if both are available at the decoder. [ABSTRACT FROM AUTHOR]

In this paper, we study and analyze the computational complexity of a software-based H.264/AVC baseline profile decoder. Our analysis is based on determining the number of basic computational operations required by a decoder to perform the key decoding subfunctions. The frequency of use of each of the required decoding subfunctions is empirically derived using bitstreams generated from two different encoders for a variety of content, resolutions and bit rates. Using the measured frequencies, estimates of the decoder time complexity for various hardware platforms can be determined. A detailed example is provided to assist readers in deriving their own time complexity estimates. We compare the resulting estimates to numbers measured for an optimized decoder on the Pentium 3 hardware platform. We then use those numbers to evaluate the dependence of the time complexity of each of the major decoder subfunctions on encoder characteristics, content, resolution and bit rate. Finally, we compare an H.264/AVC-compliant baseline decoder to a decoder that is compliant with the H.263 standard, which is currently dominant in interactive video applications. Both "C" only decoder implementations were compared on a Pentium 3 hardware platform. Our results indicate that an H.264/AVC baseline decoder is approximately 2.5 times more time complex than an H.263 baseline decoder. [ABSTRACT FROM AUTHOR]