Search

This article introduces the ISO/IEC MPEG Immersive Video (MIV) standard, MPEG-I Part 12, which is undergoing standardization. The draft MIV standard provides support for viewing immersive volumetric content captured by multiple cameras with six degrees of freedom (6DoF) within a viewing space that is determined by the camera arrangement in the capture rig. The bitstream format and decoding processes of the draft specification along with aspects of the Test Model for Immersive Video (TMIV) reference software encoder, decoder, and renderer are described. The use cases, test conditions, quality assessment methods, and experimental results are provided. In the TMIV, multiple texture and geometry views are coded as atlases of patches using a legacy 2-D video codec, while optimizing for bitrate, pixel rate, and quality. The design of the bitstream format and decoder is based on the visual volumetric video-based coding (V3C) and video-based point cloud compression (V-PCC) standard, MPEG-I Part 5.

The latest video coding standard, High Efficiency Video Coding (HEVC), uses quarter-pixel motion vector (MV) resolution for motion compensation. The adaptation of MV resolution supported by progressive MV resolution (PMVR) brings further improvement to performance by progressively adjusting the resolution according to the distance between the MV and its predictor. However, progressive adjustment of resolution by PMVR does not consider the inherent characteristics of the coding block. In this paper, we propose several ways to improve PMVR. First, we show that the performance of PMVR is correlated with the spatiotemporal characteristics of the video sequence. Then, to cope with the limitations of PMVR, we propose a flexible framework for the adaptation of MV resolution using: 1) PU size and gradient; 2) PU size, gradient, and MV components; and 3) PU size and spatiotemporal characteristics of the frames. Finally, a smart motion estimation around multiple MV predictors is performed to take full advantage of the proposed scheme. The proposed tools are implemented on top of HM-16.6. Extensive experiments and comparison with HEVC show 1.3%, 2.7%, and 1.0% average BD-Rate savings for random access, low-delay P, and low-delay B configurations, respectively.

International audience; Super-multiview video consists in a 2D arrangement of cameras acquiring the same scene and it is a well-suited format for immersive and free navigation video services. However, the large number of acquired viewpoints calls for extremely effective compression tools. View synthesis allows to reconstruct a viewpoint using nearby cameras texture and depth information. In this work we explore the potential of recent advances in view synthesis algorithms to enhance the compression performances of super-multiview video. Towards this end we consider five methods that replace one viewpoint with a synthesized view, possibly enhanced with some side information. Our experiments suggest that, if the geometry information (i.e. depth map) is reliable, these methods have the potential to improve rate-distortion performance with respect to traditional approaches, at least for some specific content and configuration. Moreover, our results shed some light about how to further improve compression performance by integrating new view-synthesis prediction tools within a 3D video encoder.

One of the most challenging goals of future immersive services is to enable the observation of a scene from any viewpoint, thus making free-navigation possible under certain constraints. In order to provide such kind of services with smooth navigation, a huge amount of views should be available on the client’s device. In particular, it is important for the case of 2D-multiview content, where cameras are positioned on a 2D grid in order to provide both horizontal and vertical parallax. This kind of content requires a large coding rate; therefore improving the compression performance of video encoders is especially relevant in this case. This paper studies how the encoder configuration affects the compression, by taking into account the spatial position of each camera. Four parameters are addressed in this work: coding order of the views, the number of reference lists, the number of reference pictures, and the ordering of pictures in the reference lists. An average of 12.0% bitrate saving is achieved for medium bitrate and 11.1% for low bitrate compared to the state of the art techniques.

International audience; With the rapid emergence of various 360 video capturing devices and head-mounted displays, providing immersive experience using 360 videos is becoming a topic of paramount interest. The current techniques face motion sickness issues, resulting in low quality of experience. One of the reason is that they do not take advantage of the parallax between the divergent views, which may be helpful to provide the correct view according to the user's head motion. In this paper, we propose to get rid of the classical ERP representation, and to synthesize arbitrary views using different divergent views together with their corresponding depths. Thus, we can exploit the parallax between the divergent views. In this context, we assess the feasibility of the depth estimation and the view synthesis using state-of-the-art techniques. Simulation results confirmed the feasibility of such a proposal, in addition to possibility to achieve sufficient visual quality for a head motion up to 0.1m from the rig, when using generated depth map for view synthesis.

International audience; 360- video is gaining a lot of interest because of the immersive feeling brought by such a technology. Several projection formats are used to represent this type of content. Equirectangular projection (ERP) is one of the most widely used projection scheme for 360 panoramic content. The main drawback of ERP is its latitude dependent sampling density unlike conventional 2D content. Consequently, conventional 2D codecs such as HEVC are not optimal for the coding of ERP projected 360 content. To cope with this dependency, this work proposes an adaptation of motion vector resolution and minimum width of the coding block depending on its latitude. Experimental results show up to 0.5% BD-rate savings for motion contained sequences with 15% encoding time reduction in random access configuration.

The latest video coding standard, HEVC, uses quarter pixel motion vector (MV) resolution for motion compensation. The adaptation of MV resolution supported by PMVR (progressive MV resolution) brings further improvement of the performance, by progressively adjusting the resolution according to the distance between the MV and the MV predictor (MVP). In this work, we propose to improve PMVR by adapting MV resolution at the prediction unit (PU) level relying on its size and its average absolute gradient. We additionally perform a smarter motion estimation around multiple MV predictors to fully take advantage of the proposed scheme. Compared to HEVC reference software (HM-16.6), the proposed method provides 1.2%, 3.2% and 1.2% average BD rate savings respectively for random access (RA), low-delay P (LP) and low-delay (LD) configurations.

International audience; Integral imaging is a technology based on plenoptic photography that captures and samples the light-field of a scene through a micro-lens array. It provides views of the scene from several angles and therefore is foreseen as a key technology for future immersive video applications. However, integral images have a large resolution and a structure based on micro-images which is challenging to encode. A compression scheme for integral images based on view extraction has previously been proposed, with average BD-rate gains of 15.7% (up to 31.3%) reported over HEVC when using one single extracted view. As the efficiency of the scheme depends on a tradeoff between the bitrate required to encode the view and the quality of the image reconstructed from the view, it is proposed to increase the number of extracted views. Several configurations are tested with different positions and different number of extracted views. Compression efficiency is increased with average BD-rate gains of 22.2% (up to 31.1%) reported over the HEVC anchor, with a realistic runtime increase.

ISO/IEC MPEG and ITU-T VCEG have recently jointly issued a new multiview video compression standard, called 3D-HEVC, which reaches unpreceded compression performances for linear,dense camera arrangements. In view of supporting future highquality,auto-stereoscopic 3D displays and Free Navigation virtual/augmented reality applications with sparse, arbitrarily arranged camera setups, innovative depth estimation and virtual view synthesis techniques with global optimizations over all camera views should be developed. Preliminary studies in response to the MPEG-FTV (Free viewpoint TV) Call for Evidence suggest these targets are within reach, with at least 6% bitrate gains over 3DHEVC technology.

New standardization activities have been recently launched by the JCT-VC experts group in order to challenge the current video compression standard H.264/AVC. Several improvements of this standard, previously integrated in the JM key technical area software, are already known and gathered in the high efficiency video coding test model. In particular, competition-based motion vector prediction has proved its efficiency. However, the targeted 50% bitrate saving for equivalent quality is not yet achieved. In this context, this paper proposes to reduce the signaling information resulting from this motion vector competition, by using data hiding techniques. As data hiding and video compression traditionally have contradictory goals, an advanced study of data hiding schemes is first performed. Then, an original way of using data hiding for video compression is proposed. The main idea of this paper is to hide the competition index into appropriately selected chroma and luma transform coefficients. To minimize the prediction errors, the transform coefficients modification is performed via a rate-distortion optimization. The proposed scheme is evaluated on several low and high resolution sequences. Objective improvements (up to 2.40% bitrate saving) and subjective assessment of the chroma loss are reported.

This paper describes a new video coding scheme based on a simplified block structure that significantly outperforms the coding efficiency of the ISO/IEC 14496-10 ITU-T H.264 advanced video coding (AVC) standard. Its conceptual design is similar to a typical block-based hybrid coder applying prediction and subsequent prediction error coding. The basic coding unit is an 8 × 8 block for inter, and an 8 × 8 or a 16 × 16 block for intra, instead of the usual 16 × 16 macroblock. No larger block sizes are considered for prediction and transform. Based on this simplified block structure, the coding scheme uses simple and fundamental coding tools with optimized encoding algorithms. In particular, the motion representation is based on a minimum partitioning with blocks sharing motion borders. In addition, compared to AVC, the new and improved coding techniques include: block-based intensity compensation, motion vector competition, adaptive motion vector resolution, adaptive interpolation filters, edge-based intra prediction and enhanced chrominance prediction, intra template matching, larger trans forms and adaptive switchable transforms selection for intra and inter blocks, and nonlinear and frame-adaptive de-noising loop filters. Finally, the entropy coder uses a generic flexible zero tree representation applied to both motion and texture data. Attention has also been given to algorithm designs that facilitate parallelization. Compared to AVC, the new coding scheme offers clear benefits in terms of subjective video quality at the same bit rate. Objective quality improvements are equally significant. At the same quality, an average bit-rate reduction of 31% compared to AVC is reported.

Le standard de compression video H.264/AVC obtient des gains significatifs par rapport a ses predecesseurs en diminuant fortement l'entropie des residuels de texture a l'aide, notamment, d'une augmentation du nombre de possibilites de codage. Les travaux pour une future norme ont deja commence au sein de chacun des deux groupes a l'origine de la norme H.264/AVC. La methode de codage de l'information de mouvement par competition de prediction de vecteurs a ete proposee dans le cadre des travaux exploratoires du groupe VCEG. Cette methode visant a reduire la quantite d'information de mouvement a ete integree dans le logiciel JM-KTA visant a preparer le nouveau standard. Dans cet article, une amelioration de cette methode est proposee, permettant de limiter le debit des indices de predicteurs. Elle est basee sur les caracteristiques de la fonction de selection du meilleur predicteur qui est reproduite au decodeur. Cette methode, testee dans le JM-KTA, elimine 10 % des indices de predicteurs et reduit ainsi le debit de l'information de mouvement.

The H.264/MPEG4-AVC video coding standard has achieved a higher coding efficiency compared to its predecessors. The significant bitrate reduction is mainly obtained by efficient motion compensation tools, as variable block sizes, multiple reference frames, 1/4-pel motion accuracy and powerful prediction modes (e.g., SKIP and DIRECT). These tools have contributed to an increased proportion of the motion information in the total bit- stream. To achieve the performance required by the future ITU-T challenge, namely to provide a codec with 50% bitrate reduction compared to the current H.264, the reduction of this motion information cost is essential. This paper proposes a competing framework for better motion vector coding and SKIP mode. The predictors for the SKIP mode and the motion vector predictors are optimally selected by a rate-distortion criterion. These methods take advantage from the use of the spatial and the temporal redundancies in the motion vector fields, where the simple spatial median usually fails. An adaptation of the temporal predictors according to the temporal distances between motion vector fields is also described for multiple reference frames and B-slices options. These two combined schemes lead to a systematic bitrate saving on Baseline and High profile, compared to an H.264/MPEG4-AVC standard codec, which reaches up to 45%.

International audience; Integral imaging is a glasses-free 3D video technology that captures a light-field representation of a scene. This representation eliminates many of the limitations of current stereoscopic and autostereoscopic techniques. However, integral images have a large resolution and a structure based on microimages which is challenging to encode. In this paper a compression scheme for integral images based on view extraction is proposed. Average BD-rate gains of 15.7% and up to 31.3% are reported over HEVC. Parameters of the proposed coding scheme can take a large range of values. Results are first provided with an exhaustive search of the best configuration. Then an RD criterion is proposed to avoid exhaustive search methods, saving runtime while preserving the gains. Finally, additional runtime savings are reported by exploringhow the different parameters interact.

Super Multi-View (SMV) video content is composed of tens or hundreds of views that provide a light-field representation of a scene. This representation allows a glass-free visualization and eliminates many causes of discomfort existing in current available 3D video technologies. Efficient video compression of SMV content is a key factor for enabling future 3D video services. This paper first compares several coding configurations for SMV content and several inter-view prediction structures are also tested and compared. The experiments mainly suggest that large differences in coding efficiency can be observed from one configuration to another. Several ratios for the number of coded and synthesized views are compared, both objectively and subjectively. It is reported that view synthesis significantly affects the coding scheme. The amount of views to skip highly depends on the sequence and on the quality of the associated depth maps. Reported ranges of bitrates required to obtain a good quality for the tested SMV content are realistic and coherent with future 4K/8K needs. The reliability of the PSNR metric for SMV content is also studied. Objective and subjective results show that PSNR is able to reflect increase or decrease in subjective quality even in the presence of synthesized views. However, depending on the ratio of coded and synthesized views, the order of magnitude of the effective quality variation is biased by PSNR. Results indicate that PSNR is less tolerant to view synthesis artifacts than human viewers. Finally, preliminary observations are initiated. First, the light-field conversion step does not seem to alter the objective results for compression. Secondly, the motion parallax does not seem to be impacted by specific compression artifacts. The perception of the motion parallax is only altered by variations of the typical compression artifacts along the viewing angle, in cases where the subjective image quality is already low. To the best of our knowledge, this paper is the first to carry out subjective experiments and to report results of SMV compression for light-field 3D displays. It provides first results showing that improvement of compression efficiency is required, as well as depth estimation and view synthesis algorithms improvement, but that the use of SMV appears realistic according to next generation compression technology requirements. HighlightsStudy of the impact of compression on subjective quality for lightfield SMV content.To the best of our knowledge, this paper is the first to report results of this kind.Several SMV coding configurations are compared both objectively and subjectively.Compression efficiency, depth estimation and view synthesis require improvements.SMV appears realistic according to next generation compression technology requirements.

Motion parallax is a key cue in the perception of the depth that current 3D stereoscopy and auto-stereoscopy technologies are not able to reproduce. Integral imaging and Super Multi-View video (SMV) are 3D technologies that allow creating a light-field representation of a scene with a smooth full motion parallax (i.e., in horizontal and vertical directions). However the large amount of data required is challenging and implies a need for new efficient coding technologies. This chapter first describes integral imaging and SMV content, acquisition, and display. Then it provides an overview of state-of-the-art methods for full parallax 3D content compression. Finally, several coding schemes are compared and a coding structure that exploits inter-view correlations in both horizontal and vertical directions is proposed. The new structure provides a rate reduction (for the same quality) up to 29.1 % when compared to a basic anchor structure. Neighboring Block Disparity Vector (NBDV) and Inter-View Motion Prediction (IVMP) coding tools are further improved to efficiently exploit coding structures in two dimensions, with rate reduction up to 4.2 % with respect to the reference 3D-HEVC encoder.

The coding efficiency of the new video coding standard, High Efficiency Video Coding (HEVC), is strongly associated with better use of spatio-temporal redundancies thanks to an increased number of competing coding modes. However, this competition involves a massive increase in signaling bitrate which becomes a possible limit for the next generation of encoder. Thispaperproposesa newcodingschemethat breakswith conventional approaches. It exploits a more complex decoder able to reproduce the choice of the encoder based on causal references, eliminating thus the need to signal coding modes and associated parameters. The general outline of this new codec and a proposed implementation are described in this paper. Experimental results under common test conditions report an average bitrate saving of 1.7% at the same quality compared to HEVC for a wide range of video sequences.

The 3D video extension of High Efficiency Video Coding (3D-HEVC) exploits texture-depth redundancies in 3D videos using intercomponent coding tools. It also inherits the same quadtree coding structure as HEVC for both components. The current software implementation of 3D-HEVC includes encoder shortcuts that speed up the quadtree construction process, but those are always accompanied by coding losses. Furthermore, since the texture and its associated depth represent the same scene, at the same time instant and view point, their quadtrees are closely linked. In this paper, an intercomponent tool is proposed in which this link is exploited to save both runtime and bits through a joint coding of the quadtrees. If depth is coded before the texture, the texture quadtree is initialized from the coded depth quadtree. Otherwise, the depth quadtree is limited to the coded texture quadtree. A 31% encoder runtime saving, a -0.3% gain for coded and synthesized views and a -1.8% gain for coded views are reported for the second method.

With the recent development of new three-dimensional (3D) multimedia services such as 3D television or free viewpoint television, a new 3D video format, called multiview video + depth (MVD) is currently being investigated. MVD allows synthesizing as many views as required at the receiver side, thus providing smooth scene transitions and the ability to experience a new 3D perspective with each different viewing point. The format introduces, alongside traditional 2D image sequences, sequences of depth maps, which must be efficiently coded to achieve good quality for the synthesized views. One approach to code depth videos is to exploit the correlations between texture and depth. In this work, we propose a new tool to code depth videos in which the texture Intra modes are inherited and used as predictors for the depth Intra modes, hence reducing the mode signaling bitrate. The tool is only used in prediction units where texture and depth Intra directions, or modes, are expected to match. Two criteria that exploit the statistical dependency between the texture and depth Intra modes are studied in this work: GradientMax and DominantAngle. Average bitrate reductions of 1.3 and 1.6% on synthesized sequences are reported for GradientMax and DominantAngle, respectively. The latter method additionally achieves 2.3% bitrate reduction on depth sequences.

The up-and-coming extension of HEVC for 3D video (3D-HEVC) includes various tools to exploit different redundancies in a 3D video signal. Inter-view redundancies are in particular exploited using Inter-View Motion Prediction (IVMP) and Inter-View Residual Prediction (IVRP). Both of these tools compensate disparity-wise the current prediction unit (PU) in order to find its corresponding PU in a base view, from which some prediction information for the current PU is retrieved. The disparity vector (DV) used for disparity compensation is currently derived using a neighboring search process (NBDV) for a DV across spatial and temporal neighbors. The first DV found is selected as the final DV used in IVMP and IVRP, with no guarantee of optimality. In this paper, the NBDV derivation process is changed: all found DVs from different neighbors are stored in a list. Redundant vectors in this list are removed, and a median computation on the remaining vectors is performed. The resulting DV is set as the DV used for IVMP. Average bitrate reductions of 0.6% and 0.8% for the two dependent views and 0.2% on synthesized views are reported with only a slight increase in encoder and decoder runtimes.

A test model for an HEVC-based 3D video coding standard (3D-HEVC) has recently been drafted. 3D-HEVC exploits inter-view redundancies by including disparity-compensated prediction (DCP) for efficient dependent view coding. It also uses the Merge coding mode to reduce the cost of motion / disparity parameters. However, the candidates in the Merge list are mostly temporal motion vectors. DCP does not often benefit from accurate predictors and is thus costly. Consequently, motion-compensated prediction (MCP) remains largely preferred. In this paper, we propose to reduce the cost of DCP by modifying the Merge candidate list to always include a disparity vector candidate. Two methods are proposed: the new candidate is either added in the secondary or in the primary list of candidates. The latter method, which achieves average bitrate reductions of 0.6% for dependent views, and 0.2% for coded and synthesized views, was adopted in both the 3D-HEVC working draft and software.

The rate allocation tradeoff between motion vectors and transform coefficients has a major importance when it comes to efficient video compression. This paper introduces a new coding mode for an H.264/AVC-like video coder, which improves the management of this resource allocation. The proposed technique can be used within any hybrid video encoder allowing a different coding mode for any macroblock. The key tool of the new mode is the lossy coding of motion vectors, obtained via quantization: while the transformed motion-compensated residual is computed with a high-precision motion vector, the motion vector itself is quantized before being sent to the decoder, in a rate/distortion optimized way. Several problems have to be faced with in order to get an efficient implementation of the coding mode, especially the coding and prediction of the quantized motion vectors, and the selection and encoding of the quantization steps. This new coding mode improves the performance of the hybrid video encoder over several sequences at different resolutions.

International audience; 2010 appears to be the launching date for new compression activities intended to challenge the current video compression standard H.264/AVC. Several improvements of this standard are already known like competition-based motion vector prediction. However the targeted 50% bitrate saving for equivalent quality is not yet achieved. In this context, this paper proposes to reduce the signaling information resulting from this vector competition, by using data hiding techniques. As data hiding and video compression traditionally have contradictory goals, a study of data hiding is first performed. Then, an efficient way of using data hiding for video compression is proposed. The main idea is to hide the indices into appropriately selected chroma and luma transform coefficients. To minimize the prediction errors, the modification is performed via a rate-distortion optimization. Objective improvements (up to 2.3% bitrate saving) and subjective assessment of chroma loss are reported and analyzed for several sequences.

International audience; Since the finalization of the H.264/AVC standard and in order to meet the target set by both ITU-T and MPEG to define a new standard that reaches 50% bit rate reduction compared to H.264/AVC, many tools have efficiently improved the texture coding and the motion compensation accuracy. These improvements have resulted in increasing the proportion of bit rate allocated to motion information. Thus, the bit rate reduction of this information becomes a key subject of research. This paper proposes a method for motion vector coding based on an adaptive redistribution of motion vector residuals before entropy coding. Motion information is gathered to forecast a list of motion vector residuals which are redistributed to unexpected residuals of lower coding cost. Compared to H.264/AVC, this scheme provides systematic gain on tested sequences, and 2.3% in average, reaching up to 4.9% for a given sequence.

International audience; New activities have been recently launched in order to challenge the H.264/AVC standard. Several improvements of this standard are already known, however the targeted 50% bitrate saving for equivalent quality is not yet achieved. Abstract In this context, a previous work proposes to use data hiding techniques to reduce the signaling information resulting from an improvement of Inter-coding. The main idea is to hide the indices into appropriately selected chroma and luma transform coefficients. To minimize the prediction errors, the modification is performed via a rate-distortion optimization. Abstract In this paper, the scheme is extended for Intra-coding and 4:4:4 sequences in order to explore the limits highlighted by the previous study and tackle some remaining issues. A different rate-distortion optimization built on the Pareto theory is also proposed. Resulting improvements (1.5% in average) for several sequences are reported and analyzed.

The coding resources used for motion vectors (MVs) can attain quite high ratios even in the case of efficient video coders like H.264, and this can easily lead to suboptimal rate-distortion performance. In a previous paper, we proposed a new coding mode for H.264 based on the quantization of motion vectors (QMV). We only considered the case of 16x16 partitions for motion estimation and compensation. That method allowed us to obtain an improved trade-off in the resource allocation between vectors and coefficients, and to achieve better rate-distortion performances with respect to H.264. In this paper, we build on the proposed QMV coding mode, extending it to the case of macroblock partition into smaller blocks. This issue requires solving some problems mainly related to the motion vector coding. We show how this task can be performed efficiently in our framework, obtaining further improvements over the standard coding technique.

H.264/MPEG4-AVC is the latest video codec provided by the Joint Video Team, gathering ITU-T and ISO/IEC experts. Technically there are no drastic changes compared to its predecessors H.263 and MPEG-4 part 2. It however significantly reduces the bitrate and seems to be progressively adopted by the market. The gain mainly results from the addition of efficient motion compensation tools, variable block sizes, multiple reference frames, 1/4-pel motion accuracy and powerful Skip and Direct modes. A close study of the bits repartition in the bitstream reveals that motion information can represent up to 40% of the total bitstream. As a consequence reduction of motion cost is a priority for future enhancements. This paper proposes a competition-based scheme for the prediction of the motion. It impacts the selection of the motion vectors, based on a modified rate-distortion criterion, for the Inter modes and for the Skip mode. Combined spatial and temporal predictors take benefit of temporal redundancies, where the spatial median usually fails. An average 7% bitrate saving compared to a standard H.264/MPEG4-AVC codec is reported. In addition, on the fly adaptation of the set of predictors is proposed and preliminary results are provided.

H.264/MPEG4-AVC achieves higher compression gain in comparison to its predecessors H.263 and MPEG4 part 2. This gain partly results from the improvement of motion compensation tools especially the variable block size, the 1/4-pel motion accuracy and the access to multiple reference frames. A particular mode among all Inter modes is the Skip mode. For this mode, no information is transmitted except the signaling of the mode itself. In our previous work we have proposed a competing framework for better motion vector prediction and coding, also including the Skip mode. This proposal has recently been adopted by the Video Coding Expert Group (VCEG) in the Key Technical Area-software (KTA) of H.264, which is the starting point for future ITU standardization activities. In this paper we propose an extension of this method based on the adaptation of two families of predictors for the Skip mode according to the video content and to statistical criteria. A systematic gain upon the previous method, with an average of 8.2% of bits saved compared to H.264 standard, is reported.

In this paper, a method for designing low-power video schemes is presented. Algorithms that imply a very low dissipation are required for new applications where the energy source is limited, e.g. mobile phones including a camera and video features. Whereas it can be observed that video standards are mainly designed around coding efficiency, we propose to take into account power consumption characteristics directly when designing the algorithm. More precisely, we give some guidelines for the design of low-power video codecs in the scope of modern hardware architectures and we introduce the notion of power scalability. We present an original encoder based on so-called 'Collocated Motion Estimation' designed using the proposed methodology. Experimental results show that we remain close to the coding efficiency of the reference H.264 baseline encoder while the power consumption is largely reduced in our solution. Moreoever this encoder is scalable in memory transfer and computational complexity.

In this paper, we present a novel H.264 video decoder where memory transfers and energy consumption are significantly reduced. A power-friendly solution is indeed required in mobile applications, where autonomy is a key feature. Whereas low-power design is usually regarded as a pure architectural and implementation problem, our solution is based on an algorithmic approach. We first observe that in modern hardware architectures, computational complexity plays a second role in terms of energy dissipation, which is dominated by memory transfers. We thus identify the motion compensation module as the most consuming part of the standard H.264 decoder, due to its numerous memory accesses to the reference frame(s). Through an algorithmic modification that uses concurrent embedded compression techniques and relies on the data structure of the memory, we reduce memory transfers, and hence power dissipation, while maintaining the visual quality. Experimental results prove that the new method is close to the reference H.264 baseline decoder both in terms of objective and subjective measurements. In the meantime, memory transfers have been reduced by 55% in average, implying power savings which lengthen the battery life of the mobile device, increase the reliability of the chip and lower production costs.

In this paper, we introduce a new decoding algorithm for DCT-based video encoders, such as Motion JPEG (M-JPEG), H26x, or MPEG. This algorithm considers not only the compression artifacts but also the ones due to transmission, acquisition or storage of the video. The novelty of our approach is to jointly tackle these two problems, using a variational approach. The resulting decoder is object-based, allowing independent and adaptive processing of objects and backgrounds, and considers available information provided by the bitstream, such as quantization steps, and motion vectors. Several experiments demonstrate the efficiency of the proposed method. Objective and subjective quality assessment methods are used to evaluate the improvement upon standard algorithms, such as the deblocking and deringing filters included in MPEG-4 postprocessing.

This paper presents FaDA (Fast Deblocking Algorithm), a low complexity post-processing algorithm that improves the visual quality of MPEG-4 low bit rate videos displayed on mobile devices. Most of the algorithms proposed up to now to enhance the visual quality after the decoder are far too complex for this kind of devices, and unfortunately deblocking algorithms efficiency is tightly related to their complexity. FaDA achieves complexity reduction while maintaining quality thanks to two principles: 1- The artifact correction is driven by Human Visual System properties, 2- A particular attention has been paid on the design of the most consuming parts of the algorithm, following implementation guidelines. We present a complete complexity evaluation that considers not only computational charge, but also number and locality of memory accesses, data dependency, and internal data size requirements. Critical steps of the algorithm were implemented in C and optimized on a TriMedia DSP. The complexity evaluation reveals a reduction by a factor ranging between 2 and 3.5, compared to the reference MPEG-4 informative deblocking, depending on architecture and implementation. Extensive subjective tests performed on both mobile LCD screen and high resolution monitor show that FaDA average visual quality is equivalent to the reference algorithm.

This paper introduces an optimal decoding scheme for the baseline Joint Photographers Expert Group (JPEG) standard. In particular, it deals with the minimization of a half-quadratic criterion which takes into account observed data, a priori knowledge of the solution, and precise spatial location of blocking artifacts. The method considers, at the same time, information in the original spatial domain, in the DCT domain, and in the spatio-frequency (DWT) domain. A model of the quantizer is also included. The method reduces blocking and ringing artifacts, resulting in improved peak signal-to-noise ratio performance as well as greater visual quality.

Standard DCT -based video coding techniques perform good results in terms of data compaction, making feasible the use of digital video in several application frameworks. The price to pay is the introduction of annoying visual distortions/artefacts in the reconstructed video. The lower the encoding bit-rate, the larger the number of artefacts. Post-processing is a practical solution that achieves a visual enhancement of the compressed images after decoding. Some of the artefacts, such as blocking (tiled-effect aspect) and ringing (ghost effect) have already been widely studied.

In this paper, we propose an object-based variational approach for the decoding of MJPEG and MPEG1-2 video sequences. This new method improves the visual quality of the reconstructed sequence by reducing the impact of cell losses, due to the transmission of the sequences over lossy packet networks. Generally, two kinds of approaches are suggested to tackle this problem: a preventative 'network' approach tends to provide a more robust bit-stream in order to help the recovery of the lost data, for instance by adding redundancy to the bitstream, and a concealment 'image' approach, that tends to reduce visual impact resulting from the packet loss. The proposed method combines 'network' information provided by the transmission protocol, with an adapted 'image' decoding algorithm. Several experimental results demonstrate the efficiency of the decoding method: lost areas are largely recovered, compared to a standard decoding, both on the background and on the objects.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper, we propose an object-based variational approach for the decoding of DV, M-JPEG, and MPEG1-2 video sequences. This new method improves the visual quality by considering simultaneously two kind of artifacts: artifacts due to compression, such as blocking effects and quantization noise, and other defects due to acquisition, transmission, or storage, such as dropouts and banding. Generally, methods improving the visual quality in video sequences consider these two kind of artifacts separately, and consist most of the time in post-processing techniques, applied to a single kind of artifact. The proposed method adopts a global approach for the decoding. It deals with the minimization of half-quadratic criteria, which allow the simultaneous estimation and restoration of backgrounds, on one side, and the moving objects detection on the other side. Each background and each object are processed separately, according to their spatial and temporal properties, to remove effectively blocking effects, quantization noise, and dropouts. Several experimental results are presented in this paper. They demonstrate the efficiency of the decoding method: blocking effects are largely removed and missing data are significantly reduced, compared to the standard decoding, resulting in a greater visual quality of the sequence.

This paper addresses several downsides of the system under development in MPEG-I for coding and transmission of immersive media. We present a solution, which enables Depth-Image-Based Rendering for immersive video applications, while lifting the requirement of transmitting depth information. Instead, we estimate the depth information on the client-side from the transmitted views. The approach leads to an impressive rate saving (37.3% in average). Preserving perceptual quality in terms of MS-SSIM of synthesized views, it yields to 24.6% rate reduction for the same quality of reconstructed views after residue transmission under the MPEG-I common test conditions. Simultaneously, the required pixel rate, i.e. the number of pixels processed per second by the decoder, is reduced by 50% for any test sequence. To the author’s knowledge, this is the first time that such an approach is under consideration in the context of immersive video coding.

Super Multi-View (SMV) video is a key enabler for future 3D video services that allows a glasses-free visualization and eliminates many causes of discomfort existing in current available 3D video technologies. SMV video content is composed of tens or hundreds of views, that can be aligned in horizontal only or both horizontal and vertical directions, providing respectively horizontal parallax or full parallax. This paper compares several coding schemes and coding orders, and proposes a coding structure that exploits inter-view correlations in the two directions, providing BD-rate gains up to 29.1% when compared to a basic anchor structure. Additionally, Neighboring Block Disparity Vector (NBDV) and Inter-View Motion Prediction (IVMP) coding tools are further improved to efficiently exploit coding structures in two dimensions, with BD-rate gains up to 4.2% reported over the reference 3D-HEVC encoder.