Search

There exist a variety of ways to represent 3D content, including stereo and multiview video, as well as frame-compatible and depth-based video formats. There are also a number of compression architectures and techniques that have been introduced in recent years. This paper provides an overview of relevant 3D representation and compression formats. It also analyzes some of the merits and drawbacks of these formats considering the application requirements and constraints imposed by different storage and transmission systems. [ABSTRACT FROM AUTHOR]

A depth image is 3-D information used for virtual view synthesis in 3-D video system. In depth coding, the object boundaries are hard to compress and severely affect the rendering quality since they are sensitive to coding errors. In this paper, we propose a depth boundary reconstruction filter and utilize it as an in-loop filter to code the depth video. The proposed depth boundary reconstruction filter is designed considering occurrence frequency, similarity, and closeness of pixels. Experimental results demonstrate that the proposed depth boundary reconstruction filter is useful for efficient depth coding as well as high-quality 3-D rendering. [ABSTRACT FROM PUBLISHER]

Abstract: We propose a rate-distortion-optimized framework that incorporates view synthesis for improved prediction in multiview video coding. In the proposed scheme, auxiliary information, including depth data, is encoded and used at the decoder to generate the view synthesis prediction data. The proposed method employs optimal mode decision including view synthesis prediction, and sub-pixel reference matching to improve prediction accuracy of the view synthesis prediction. Novel variants of the skip and direct modes are also presented, which infer the depth and correction vector information from neighboring blocks in a synthesized reference picture to reduce the bits needed for the view synthesis prediction mode. We demonstrate two multiview video coding scenarios in which view synthesis prediction is employed. In the first scenario, the goal is to improve the coding efficiency of multiview video where block-based depths and correction vectors are encoded by CABAC in a lossless manner on a macroblock basis. A variable block-size depth/motion search algorithm is described. Experimental results demonstrate that view synthesis prediction does provide some coding gains when combined with disparity-compensated prediction. In the second scenario, the goal is to use view synthesis prediction for reducing rate overhead incurred by transmitting depth maps for improved support of 3DTV and free-viewpoint video applications. It is assumed that the complete depth map for each view is encoded separately from the multiview video and used at the receiver to generate intermediate views. We utilize this information for view synthesis prediction to improve overall coding efficiency. Experimental results show that the rate overhead incurred by coding depth maps of varying quality could be offset by utilizing the proposed view synthesis prediction techniques to reduce the bitrate required for coding multiview video. [Copyright &y& Elsevier]

In personal communications systems, it is an emerging necessity to adapt services prior to delivery according to up-to-date customer profiles and mobility behaviors. Therefore, it is important to determine distinct mobility descriptors that would successfully model the entire sample space of different mobility characteristics. The aim of this paper is to define unique descriptors that are to be extracted from mobility patterns of a single mobile user and to investigate the level of distinctness of each defined descriptor. We finally show how the studied descriptors can be used in service adaptation. [Copyright &y& Elsevier]

This paper describes rate-control algorithms that consider the tradeoff between coded quality and temporal rate. We target improved coding efficiency for both frame-based and object-based video coding. We propose models that estimate the rate-distortion characteristics for coded frames and objects, as well as skipped frames and objects. Based on the proposed models, we propose three types of rate-control algorithms. The first is for frame-based coding, in which the distortion of coded frames is balanced with the distortion incurred by frame skipping. The second algorithm applies to object-based coding, where the temporal rate of all objects is constrained to be the same, but the bit allocation is performed at the object level. The third algorithm also targets object-based coding, but in contrast to the second algorithm, the temporal rates of each object may vary. The algorithm also takes into account the composition problem, which may cause holes in the reconstructed frame when objects are encoded at different temporal rates. We propose a solution to this problem that is based on first detecting changes in the shape boundaries over time at the encoder, then employing a hole detection and recovery algorithm at the decoder. Overall, the proposed algorithms are able to achieve the target bit rate, effectively code frames and objects with different temporal rates, and maintain a stable buffer level. [ABSTRACT FROM AUTHOR]

The purpose of this paper it to explore the relationship between the rate-distortion characteristics of multiscale binary shape and Markov random field (MRF) parameters. For coding, it is important that the input parameters that will be used to define this relationship be able to distinguish between the same shape at different scales, as well as different shapes at the same scale. In this work, we consider an MRF model, referred to as the Chien model, which accounts for high-order spatial interactions among pixels. We propose to use the statistical moments of the Chien model as input to a neural network to accurately predict the rate and distortion of the binary shape when coded at various scales. [ABSTRACT FROM AUTHOR]

This paper proposes an optimal rate-allocation scheme for fine-granular scalability (FGS) coded bitstreams that can achieve constant quality reconstruction of frames under a dynamic rate budget constraint. In doing so, we also aim to minimize the overall distortion at the same time. To achieve this, we propose a novel rate-distortion (R-D) labeling scheme to characterize the R-D relationship of the source coding process. Specifically, sets of R-D points are extracted during the encoding process and linear interpolation is used to estimate the actual R-D curve of the enhancement-layer signal. The extracted R-D information is then used by an enhancement-layer transcoder to determine the bits that should be allocated per frame. A sliding-window-based rate-allocation method is proposed to realize constant quality among frames. This scheme is first considered for a single FGS-coded source, then extended to operate on multiple sources. With the proposed scheme, the rate allocation can be performed in a single pass; hence, the complexity is quite low. Experimental results confirm the effectiveness of the proposed scheme under static and dynamic bandwidth conditions. [ABSTRACT FROM AUTHOR]

This paper discusses the problem of reduced-resolution transcoding of compressed video bitstreams. An analysis of drift errors is provided to identify the sources of quality degradation when transcoding to a lower spatial resolution. Two types of drift error are considered: a reference picture error, which has been identified in previous works, and error due to the noncommutative property of motion compensation and down-sampling, which is unique to this work. To overcome these sources of error, four novel architectures are presented. One architecture attempts to compensate for the reference picture error in the reduced resolution, while another architecture attempts to do the same in the original resolution. We present a third architecture that attempts to eliminate the second type of drift error and a final architecture that relies on an intrablock refresh method to compensate for all types of errors. In all of these architectures, a variety of macroblock level conversions are required, such as motion vector mapping and texture down-sampling. These conversions are discussed in detail. Another important issue for the transcoder is rate control. This is especially important for the intra-refresh architecture since it must find a balance between number of intrablocks used to compensate for errors and the associated rate-distortion characteristics of the low-resolution signal. The complexity and quality of the architectures are compared. Based on the results, we find that the intra-refresh architecture offers the best tradcoff between quality and complexity and is also the most flexible. [ABSTRACT FROM AUTHOR]

Presents a study on the various video transcoding architectures that perform spatial resolution reduction. Introduction to transcoding in consumer electronics; Description of the algorithmic and processor-specific optimizations considered; Analysis of the architectures in terms of their complexity and quality; Conclusions.

Introduces a framework for video content delivery based on transcoding of multiple video objects. Manipulation of object-based video content; Introduction of algorithms capable of manipulating video bit streams; Discussion on the degrees of freedom within an object-based transcoder.

Deals with a study which presented a video browsing system that extracts features and other content-description from compressed video. Requirements for a video browsing system; Overview of the proposed architecture; Experimental results; Concluding remarks.

Describes techniques for implementing a low-cost video decoder that can decode an high definition television (HDTV) bitstream and display the signal at lower resolutions. Key algorithms used in the techniques; Concept of frequency synthesis down-conversion; Expression for the optimal set of low resolution.

Describes an algorithm that can achieve a constant bit rate when coding multiple video objects. Background on the pre- and postencoding stage of the algorithm; Tradeoff between texture and shape coding accuracy; Block structures of multiple video objects and single video object.

Proposes an efficient memory saving architecture for an Associated Television (ATV) decoder. Overview of the MPEG-2 video decoder; Architecture of the memory efficient decoder; Simulation results; Conclusions.

In the MPEG-2 Test Model 5, the down-conversion of an interlaced sequence is obtained by prefiltering and subsampling each field of the image sequence after it has been fully decoded. Although the quality is very good, the cost of such a system is quite high owing to large memory requirements. As a result, low-resolution decoders have been proposed to reduce some of the costs incurred by this scheme. Here, incoming Discrete Cosine Transform (DCT) blocks are subject to a down-conversion process within the decoding loop; hence, the motion compensation is performed using the down-converted images. In past work, it has been proven that the optimal filters for performing this motion compensation are intimately related to the method of down-conversion. Therefore, the choice of down-conversion filter is viewed as the primary variable affecting the quality of the down-converted sequence when such an optimal motion compensation scheme is considered. In the conventional method of frequency domain down-conversion, the 4 × 4 low-frequency coefficients are extracted from each 8 × 8 block. Two problems arise from this method: First, the discarding of high frequency data will introduce a disturbing amount of drift; and second, severe blocking artifacts will result in areas of large motion. To remedy the drift problem, a new method of down-conversion which better preserves high-frequency data is presented. This method is referred to as frame-based frequency synthesis. Then, to overcome the blocking artifacts, the previous method is extended to a field-based frequency synthesis method. Our simulation results clearly indicate that the amount of drift can be significantly reduced by retaining high-frequency data and that severe blocking artifacts are eliminated by using the field-based method. In addition, the quality achieved by the proposed field-based frequency synthesis is much closer to the high-quality results produced after full-resolution decoding. © 1998 John Wiley & Sons, Inc. Int J Imaging Syst Technol, 9, 274–282, 1998 [ABSTRACT FROM AUTHOR]

Advanced multiview video systems are able to generate intermediate viewpoints of a 3-D scene. To enable low-complexity free view generation, texture and its associated depth are used as input data for each viewpoint. To improve the coding efficiency of such content, view synthesis prediction (VSP) is proposed to further reduce interview redundancy in addition to traditional disparity compensated prediction. This paper describes and analyzes rate-distortion optimized VSP designs, which were adopted in the 3-D extensions of both Advanced Video Coding (AVC) and High Efficiency Video Coding (HEVC). In particular, we propose a novel backward-VSP scheme using a derived disparity vector, as well as efficient signalling methods in the context of AVC and HEVC. In addition, we put forward a novel depth-assisted motion vector prediction method to optimize the coding efficiency. A thorough analysis of coding performance is provided using different VSP schemes and configurations. Experimental results demonstrate average bit rate reductions of 2.5% and 1.2% in AVC and HEVC coding frameworks, respectively, with up to 23.1% bit rate reduction for dependent views. [ABSTRACT FROM AUTHOR]

Virtual view synthesis is one of the most important techniques to realize free viewpoint television and three-dimensional (3D) video. In this article, we propose a view synthesis method to generate high-quality intermediate views in such applications and new evaluation metrics named as spatial peak signal-to-noise ratio and temporal peak signal-to-noise ratio to measure spatial and temporal consistency, respectively. The proposed view synthesis method consists of five major steps: depth preprocessing, depth-based 3D warping, depth-based histogram matching, base plus assistant view blending, and depth-based hole-filling. The efficiency of the proposed view synthesis method has been verified by evaluating the quality of synthesized images with various metrics such as peak signal-to-noise ratio, structural similarity, discrete cosine transform (DCT)-based video quality metric, and the newly proposed metrics. We have also confirmed that the synthesized images are objectively and subjectively natural. © 2010 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 20, 378-390, 2010 [ABSTRACT FROM AUTHOR]

In this paper, we attempt to provide a comprehensive and high-level review of audio-visual features that can be extracted from the standard compressed domains, such as MPEG-1 and MPEG-2. The paper is motivated by the myriad of active research works in extraction and application of compressed-domain features in various fields, such as indexing, filtering, and manipulation. Compressed-domain approaches avoid expensive computation and memory requirements involved in decoding and/or re-encoding. Selected features are categorized into four groups—spatial visual (e.g., color, texture, edge, shape), motion (e.g., motion field, trajectory), audio (e.g., energy, spectral features, pitch), and coding (e.g., bit rate, frame/block type). For each feature, we briefly discuss the extraction methods, computational complexity, potential effectiveness in applications, and possible limitations caused by compress-domain approaches. Finally, we also discuss the possibilities of extracting some important MPEG-7 visual and audio descriptors directly from the compressed domain. [Copyright &y& Elsevier]

To reduce the cost of storing, processing, and visualizing a large-scale point cloud, we propose a randomized resampling strategy that selects a representative subset of points while preserving application-dependent features. The strategy is based on graphs, which can represent underlying surfaces and lend themselves well to efficient computation. We use a general feature-extraction operator to represent application-dependent features and propose a general reconstruction error to evaluate the quality of resampling; by minimizing the error, we obtain a general form of optimal resampling distribution. The proposed resampling distribution is guaranteed to be shift-, rotation- and scale-invariant in the three-dimensional space. We then specify the feature-extraction operator to be a graph filter and study specific resampling strategies based on all-pass, low-pass, high-pass graph filtering and graph filter banks. We validate the proposed methods on three applications: Large-scale visualization, accurate registration, and robust shape modeling demonstrating the effectiveness and efficiency of the proposed resampling methods. [ABSTRACT FROM AUTHOR]

The High Efficiency Video Coding (HEVC) standard has recently been extended to support efficient representation of multiview video and depth-based 3D video formats. The multiview extension, MV-HEVC, allows efficient coding of multiple camera views and associated auxiliary pictures, and can be implemented by reusing single-layer decoders without changing the block-level processing modules since block-level syntax and decoding processes remain unchanged. Bit rate savings compared with HEVC simulcast are achieved by enabling the use of inter-view references in motion-compensated prediction. The more advanced 3D video extension, 3D-HEVC, targets a coded representation consisting of multiple views and associated depth maps, as required for generating additional intermediate views in advanced 3D displays. Additional bit rate reduction compared with MV-HEVC is achieved by specifying new block-level video coding tools, which explicitly exploit statistical dependencies between video texture and depth and specifically adapt to the properties of depth maps. The technical concepts and features of both extensions are presented in this paper. [ABSTRACT FROM PUBLISHER]

This paper surveys recent trends and perspectives in distributed video coding. More specifically, the status and potential benefits of distributed video coding in terms of coding efficiency, complexity, error resilience, and scalability are reviewed.Multiview video and applications beyond coding are also considered. In addition, recent contributions in these areas, more thoroughly explored in the papers of the present special issue, are also described. [ABSTRACT FROM AUTHOR]

A fuzzy filter adaptive to both sample's activity and the relative position between samples is proposed to reduce the artifacts in compressed multidimensional signals. For JPEG images, the fuzzy spatial filter is based on the directional characteristics of ringing artifacts along the strong edges. For compressed video sequences, the motion compensated spatiotemporal filter (MCSTF) is applied to intraframe and interframe pixels to deal with both spatial and temporal artifacts. A new metric which considers the tracking characteristic of human eyes is proposed to evaluate the flickering artifacts. Simulations on compressed images and videos show improvement in artifact reduction of the proposed adaptive fuzzy filter over other conventional spatial or temporal filtering approaches. [ABSTRACT FROM AUTHOR]

This paper presents an HDTV video decoder core that is able to decode MPEG-2, MPEG-4 A VC and VC-I formats and is fully compatible with the Blu-ray Disc standard. The core has two major features which achieve low-cost hardware implementation for three different standards. First, a novel re-configurable architecture is adopted to realize reduced hardware for three different variable length coding tables. A hybrid architecture that consists of a cell array and coefficients memory is introduced By considering the trade-off between performance and cost, the cell array can compare a variable number of bits in a flexible manner. Second, a data compression method suitable for all video decoding standards is applied to reduce memory data usage and access bandwidth. A compression syntax based on Run-Level encoding with Exp-Golomb tables is used to achieve both low compression workload and reasonable memory cost. The core is implemented by top-down HDL basis approach, and the circuit volume is 1.5Mgates with 9Onm CMOS technology and operation clock.frequency is 162MHz for 1080i at 30fr/s. [ABSTRACT FROM AUTHOR]

This work investigates the use of the 3D dual-tree discrete wavelet transform (DDWT) for video coding. The 3D DDWT is an attractive video representation because it isolates image patterns with different spatial orientations and motion directions and speeds in separate subbands. However, it is an overcomplete transform with 4 : 1 redundancy when only real parts are used. We apply the noise-shaping algorithm proposed by Kingsbury to reduce the number of coefficients. To code the remaining significant coefficients, we propose two video codecs. The first one applies separate 3D set partitioning in hierarchical trees (SPIHT) on each subset of the DDWT coefficients (each forming a standard isotropic tree). The second codec exploits the correlation between redundant subbands, and codes the subbands jointly. Both codecs do not require motion compensation and provide better performance than the 3D SPIHT codec using the standard DWT, both objectively and subjectively. Furthermore, both codecs provide full scalability in spatial, temporal, and quality dimensions. Besides the standard isotropic decomposition, we propose an anisotropic DDWT, which extends the superiority of the normal DDWT with more directional subbands without adding to the redundancy. This anisotropic structure requires significantly fewer coefficients to represent a video after noise shaping. Finally, we also explore the benefits of combining the 3D DDWT with the standard DWT to capture a wider set of orientations. [ABSTRACT FROM AUTHOR]