Search

This paper presents the overview and rationale behind the Decoder-Side Depth Estimation (DSDE) mode of the MPEG Immersive Video (MIV) standard, using the Geometry Absent profile, for efficient compression of immersive multiview video. A MIV bitstream generated by an encoder operating in the DSDE mode does not include depth maps. It only contains the information required to reconstruct them in the client or in the cloud: decoded views and metadata. The paper explains the technical details and techniques supported by this novel MIV DSDE mode. The description additionally includes the specification on Geometry Assistance Supplemental Enhancement Information which helps to reduce the complexity of depth estimation, when performed in the cloud or at the decoder side. The depth estimation in MIV is a non-normative part of the decoding process, therefore, any method can be used to compute the depth maps. This paper lists a set of requirements for depth estimation, induced by the specific characteristics of the DSDE. The depth estimation reference software, continuously and collaboratively developed with MIV to meet these requirements, is presented in this paper. Several original experimental results are presented. The efficiency of the DSDE is compared to two MIV profiles. The combined non-transmission of depth maps and efficient coding of textures enabled by the DSDE leads to efficient compression and rendering quality improvement compared to the usual encoder-side depth estimation. Moreover, results of the first evaluation of state-of-the-art multiview depth estimators in the DSDE context, including machine learning techniques, are presented. [ABSTRACT FROM AUTHOR]

Existing LiDAR point cloud compression (PCC) methods tend to treat compression as a fidelity issue, without sufficiently addressing its machine perception aspect. The latter issue is often encountered by the decoder agents that might aim to conduct scene-understanding related tasks only, such as computing the localization information. For tackling this challenge, a novel LiDAR PCC system is proposed to compress the point cloud geometry, which contains a back channel for allowing the decoder to initiate such request to the encoder. The key success of our PCC method lies in our proposed semantic prior representation (SPR) and its lossy encoding algorithm with variable precision to generate the final bitstream; the entire process is fast and achieves real-time performance. Note that our SPR is a compact and effective representation of three-dimensional (3D) input point clouds, and it consists of labels, predictions, and residuals. These information can be generated by first exploiting a scene-aware object segmentation to a set of 2D range images (frames) individually, which were generated from the 3D point clouds via a projection process. Based on the generated labels, the pixels associated with those moving objects are considered as noisy information and should be removed for not only saving bit budget on transmission but also, most importantly, improving the accuracy of localization computed at the decoder. Experimental results conducted on the commonly-used test dataset have shown that our proposed system outperforms the MPEG’s G-PCC (TMC13-v14.0) in a large bitrate range. In fact, the performance gap will become even larger when more and/or large moving objects are involved in the input point clouds. [ABSTRACT FROM AUTHOR]

IP traffic has increased significantly in recent years, and it is expected that this progress will continue. Recent studies report that the viewing of online video content accounts for a share of 1% of the global greenhouse gas emissions. To reduce the data traffic of video streaming, the new standard Versatile Video Coding (VVC) has been finalized in 2020. In this paper, the energy efficiency of two different VVC decoders is analyzed in detail. Furthermore, we propose a design space exploration that uses an algorithm based on a greedy strategy to derive coding tool profiles that optimize the energy demand of the decoder. We show that the algorithm derives optimal coding tool profiles for a subset of coding tools. Additionally, we propose profiles that reduce the energy demand of VVC decoders and provide energy savings of more than 50% for sequences with 4K resolution. Thereby, we will also show that the proposed profiles can have a lower decoding energy demand than comparable HEVC-encoded bit streams while also having a significantly lower bit rate. [ABSTRACT FROM AUTHOR]

The scalable extension of High Efficiency Video Coding, SHVC can code Ultra High-Definition (UHD) video, including 360-degree video for various devices to serve a single bitstream with different display resolutions and qualities. To improve the SHVC compression efficiency, this paper proposes a novel intra and inter-frame coding scheme by first separating the common/visually important information and then applying cuboid-based variable size block partitioning and coding process for the common/visually important information in the base layer. In cuboid-based partitioning a video frame is partitioned into arbitrary shaped rectangular regions, known as cuboids, based on the distribution of relatively homogeneous pixel values. As the cuboid adopts a variable block partitioning based on the homogeneity of the data value, the partitioned blocks have better alignment with the object boundary. Moreover, in the cuboid coding process, only the partitioning tree information and a single value for each block need to be coded which takes lower number of bits and computational time compared to the traditional SHVC base layer. To verify the performance of the proposed method we embedded the proposed scheme as a base layer into the standard SHVC reference software and used several popular UHD/360-degree videos. The experimental results indicate that the proposed scalable coding strategy achieves an average of 14.04% BD-Rate reduction and 0.61 dB BD-PSNR gain for UHD/360-video compared to the operation points provided by an SHVC conforming encoder. [ABSTRACT FROM AUTHOR]

Fast-growing intelligent media processing applications demand efficient processing throughout the processing chain from the edge to the cloud, and the complexity bottleneck usually lies in the parallel decoding of multiple-channel compressed bitstreams before analyzing. This occurs because the traditional media coding scheme generates a binary stream without a semantic structure, which is unable to be operated directly at the bitstream level to support different tasks such as classification, recognition, detection, etc. Therefore, in this article, we propose a learning-based semantically structured image coding (SSIC) framework to generate a semantically structured bitstream (SSB), where each part of the bitstream represents a specific object and can be directly used for the aforementioned intelligent tasks. Specifically, we integrate an object location extraction module into the compression framework to locate and align objects in the feature domain. Then, each object together with the background is compressed separately and reorganized to form a structured bitstream to enable the analysis or reconstruction of specific objects directly from partial bitstream. Furthermore, in contrast to existing learning-based compression schemes that train the specific model for a specific bitrate, we share most of the model parameters among various bitrates to significantly reduce the model size for variable-rate compression. The experimental results demonstrate the effectiveness of the proposed coding scheme whose compression performance is comparable to existing image coding schemes, where intelligent tasks such as classification and pose estimation can be directly performed on a partial bitstream without performance degradation, significantly reducing the complexity for analyzing tasks. [ABSTRACT FROM AUTHOR]

Most of the existing coverless steganography approaches have poor robustness to geometric attacks, because these approaches use features of the entire image to map information, and these features are easy to be lost when being attacked. In order to improve the robustness against geometric attacks, we propose a coverless image steganography method based on multi-object recognition. In this scheme, we firstly use Faster RCNN to detect objects in the image data set, establish a mapping dictionary between object labels and binary sequence. Then we propose a novel mapping rule based on the filtered robust object labels for sequence generation. Therefore, an image can generate robust binary sequence through multi-objects recognition. In the transmission process, the transmitted image has not been modified, so our method can fundamentally resist steganalysis tools and avoid the attacker’s suspicions. In addition, the capacity and hiding rate of the proposed method are both considerable. Evaluations with under geometric attacks shows, on average, $3.1\times $ robustness increase over other five coverless steganography methods. Moreover, evaluations under ten noise attacks shows, on average, the robustness of our method is also excellent, which reaches 83%. [ABSTRACT FROM AUTHOR]

This paper proposes a new framework of lossless data hiding (LDH) in JPEG images. The proposed framework contains two algorithms, i.e., the optimized basic LDH and the relay transfer based extension. In the basic algorithm, we aim to preserve the filesize after data embedding. The data hiding process is optimized by variable-length-code (VLC) mapping, combination and permutation. In the extended algorithm, we focus on embedding more bits into the bitstream with a condition that the filesize increment is allowed. To decrease the filesize increment, we propose a relay transfer based algorithm to preprocess the JPEG bitstream. Subsequently, we embed data into the processed bitstream using the basic LDH. Both algorithms provide better performances than previous arts. After lossless data hiding, the marked JPEG bitstream is compliant to common JPEG decoders. Since all operations are implemented on VLCs and the Huffman codes, no distortion is generated on the image pixels. Experimental results demonstrate that the proposed approach outperforms previous methods. [ABSTRACT FROM AUTHOR]

This paper aims to present a novel accurate moving object detection method based on the conditional random field (CRF) for high efficiency video coding/H.265 compressed domain video sequences. For each block, the number of consumed bits, motion vectors (MVs), and partitioning modes for a given block is extracted from the compressed bitstream. After removing outlier MVs, compensating MVs are assigned to the I-blocks based on their neighboring blocks. The information, such as MV, partitioning mode, and bit consumption, is used in the potential functions of a CRF model which is updated for every frame to detect the objects. Then, a number of standard test video sequences are used to verify the performance of the model. The results indicated that the model can offer a precision, that is more than 90% on average for the video sequences. The proposed method offers a 1.8 speedup, compared to the latest works in the compressed domain without losing the objects in the I-frames. [ABSTRACT FROM AUTHOR]

In this paper, a novel lossy compression scheme for encrypted image based on image inpainting is proposed. In order to maintain confidentiality, the content owner encrypts the original image through a modulo-256 addition encryption and block permutation to mask image content. Then, the third party, such as a cloud server, can compress the selective encrypted image before transmitting to the receiver. During compression, encrypted blocks are categorized into four sets corresponding to different complexity degrees in plaintext domain without the loss of security. By allocating various bit rates to the encrypted blocks from different sets, flexible compression can be achieved with difference quantization. After parsing and decoding the compressed bit stream, the receiver first recovers partial encrypted pixels and then decrypts them. The other missing pixels are further recovered with the assistance of image inpainting based on a total variation model, and the final reconstructed image can be produced. Experimental results demonstrate that the proposed scheme achieves better rate-distortion performance than some of the state-of-the-art schemes. [ABSTRACT FROM AUTHOR]

This paper presents a novel fast transcoding framework to efficiently bridge the state-of-art high efficiency video coding (HEVC) standard and its screen content coding (SCC) extension to support the bitstream compatibility over legacy HEVC devices. By exploiting the side information from the SCC bitstream, fast mode and partition decisions are made to accurately translate the novel SCC modes to conventional HEVC modes based on statistical mode mapping techniques. Compared with the full-decoding-full-encoding (FDFE) solution, the proposed framework achieves on average 51% and 82% complexity reductions with 0.57% Bjøntegaard-delta rate (BD-Rate) loss and 9.74% BD-Rate gain under all-intra (AI) and low-delay (LD) configurations, respectively. Compared with the direct transcoding reusing intra mode and inter motion, the proposed mode mapping framework introduces additional 23% and 6% complexity reductions for AI and LD encoding configurations with 0.43% BD-Rate loss and 1.10% BD-Rate saving, respectively. The proposed solution is extended to support the single-input-multiple-output screen content adaptive streaming at the edge clouds, where an SCC bitstream coded in high quality is transcoded into multiple HEVC bitstreams in reduced qualities. Our proposed solution achieves on average 49% and 76% complexity reductions with 0.78% BD-Rate loss and 7.40% BD-Rate gain under AI and LD configurations, respectively. [ABSTRACT FROM AUTHOR]

New methods to improve the encoding of the connectivity and geometry of the topological surgery scheme are proposed. In connectivity compression, after obtaining the vertex and triangle spanning trees by decomposing a three-dimensional object, bits are adaptively allocated to each run of two spanning trees on a threshold basis. The threshold is the length of a binary number of the maximum run length. If a run length exceeds the threshold, it is represented by a binary number of the run length. Otherwise, it is represented by a bit sequence. Therefore, compression efficiency is enhanced through an adaptive bit allocation to each run of two spanning trees. In geometry compression, since vertices represented by three-dimensional vectors are stored according to the order of the travelling along vertex spanning tree by depth-first searching, they have geometrical closeness. The geometry compression efficiency can be improved if the local characteristics of vectors are considered. Therefore, dynamic finite state vector quantization, which has subcodebooks depending on a local characteristic of vectors, is used to encode the geometry information. As it dynamically constructs a subcodebook by predicting an input vector's state, it produces less distortion and gives better visual quality than conventional methods [ABSTRACT FROM PUBLISHER]