Search

In the three-dimensional video system, the depth image-based rendering is a key technique for generating synthesized views, which provides audiences with depth perception and interactivity. However, the inaccuracy of depth information leads to geometrical rendering position errors, and the compression distortion of texture and depth videos degrades the quality of the synthesized views. Although existing quality enhancement methods can eliminate the distortions in the synthesized views, their huge computational complexity hinders their applications in real-time multimedia systems. To this end, a residual distillation enhanced network (RDEN)-guided lightweight synthesized view quality enhancement (SVQE) method is proposed to minimize holes and compression distortions in the synthesized views while reducing the model complexity. First, a rethinking on the deep-learning-based SVQE methods is performed. Then, a feature distillation attention block is proposed to effectively reduce the distortions in the synthesized views and make the model fulfill more real-time tasks, which is a lightweight and flexible feature extraction block using an information distillation mechanism and a lightweight multi-scale spatial attention mechanism. Third, a residual feature fusion block is proposed to improve the enhancement performance by using the feature fusion mechanism, which efficiently improves the feature extraction capability without introducing any additional parameters. Experimental results prove that the proposed RDEN efficiently improves the SVQE performance while consuming few computational complexities compared with the state-of-the-art SVQE methods. [ABSTRACT FROM AUTHOR]

The depth video contains large smooth contents with sharp edges. Since the deep learning-based color video orientated intra prediction methods pay no attention to the characteristics of depth video, they are unsuitable for optimizing the coding efficiency of depth video. In this paper, a multiple resolution prediction method with deep up-sampling is proposed to promote the coding efficiency of depth video. To efficiently encode the depth blocks of different complexity, the depth block is selectively encoded at different resolutions, including $\times 1$ , $\times 1$ /2, and $\times 1$ /4 resolutions. If the block is encoded with a low-resolution (LR), the resolution of reconstructed LR depth block is recovered by an up-sampling network. To constrain the quality of both reconstructed high-resolution depth block and its synthesized view, a view synthesis distortion guidance mechanism is proposed for the up-sampling network. In addition, a distillation-based lightweight up-sampling network is proposed to reduce the computational complexity. Experimental results demonstrate that the proposed multiple resolution prediction method obtains an average of 10.84% BD-rate saving in comparison with 3D-HEVC. [ABSTRACT FROM AUTHOR]

Typical high efficiency video coding (HEVC) encoders use rate-distortion optimization (RDO) to select the best coding parameters among numerous candidates to achieve high compression efficiency. However, calculating the rate-distortion cost for each coding parameter entails high computational complexity, which limits the real-time application of these encoders. In this study, time-consuming processes, such as reconstruction and entropy coding, are removed during the calculation of the full RD cost to reduce the complexity of the encoder. Instead, RD costs are estimated for transformed coefficients by using the RD cost model, which is made up of a distortion model based on the distribution features of the transformed coefficients, a residual rate model based on the characteristics of the entropy coding module, and a simple and convenient header rate model. In addition to the full RD cost, typical encoders also adopt fast RD cost, which is composed of the sum of absolute transform differences (SATD) and header bit. To further reduce encoding complexity, three simplified SATD (SSATD) methods based on DCT and DST features are proposed to serve as substitutes for SATD in fast RD cost, thus reducing the computational complexity of SATD calculation. Experimental results show that 35.53%, 14.57%, and 14.37% of coding time can be reduced respectively under AI, RA and LDP configurations with negligible RD performance loss when applying the proposed RD cost model and the SSATD methods to HM16.7. Besides, the proposed methods also can reduce 4%–9% of encoding time and exhibits the same RD performance as state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The 3D extension of High Efficiency Video Coding significantly improves the coding efficiency of 3D video at the expense of computational complexity. This paper presents a novel fast mode decision algorithm for depth map coding based on the grayscale similarity and inter-view correlation. First, depth map grayscale similarity is adopted to judge whether the reference frame could assist the coding of the current frame. When the difference in the average grayscale between the co-located coding unit (CU) and the current CU is smaller than the similarity threshold, the depth level of the current CU will be restricted by that of the coded reference CU. Second, the grayscale similarity and inter-view correlation are jointly used for dependent views to achieve early decision on the best prediction unit (PU) mode. The mode decision procedure will be determined early when the co-located CU, which has a grayscale similarity with the current CU, selects Merge or Inter $2N \times 2N$ as the best prediction mode. Moreover, when the corresponding CU in the independent view selects Merge or Inter $2N \times 2N$ as the best prediction mode, the current CU will skip other PU modes checking based on the strong inter-view correlation. Finally, different strategies are proposed for the P-frames and B-frames of dependent views in view of the characteristics of different prediction structures. For B frames, the PU mode information of the coded independent view is utilized as reference to skip the unnecessary mode decision processes. For P frames, the spatial–temporal correlation is considered in the process of early mode decision to determine whether to choose the Merge mode or Inter $2N \times 2N$ as the best mode. Experimental results show that our proposed scheme achieves considerable time saving with negligible degradation of coding performance. [ABSTRACT FROM PUBLISHER]

The arbitrary-shape scene text detection faces enormous challenges in accuracy and speed with increasing demand in the industrial sector. Although most studies have introduced segmentation and achieved remarkable performance in text detection tasks, researchers ignored the importance of detection efficiency. This paper proposes a new Joint Multi-Space Perception Network (JMNET) for efficient scene text detection to address this issue. Based on a lightweight feature extraction backbone, we put forward two novel modules, i.e., Scale Spatial Perception Module (SSPM) and Attention Spatial Perception Module (ASPM) to enhance the expression ability of text features with low computational complexity. Moreover, we propose an Unsupervised Embedding Spatial Perception Loss function (UnESP Loss) by introducing the Euclidean distance measurement between the embeddings to overcome the ambiguity of text instance boundary, such as a small line spacing. In this way, text embedding learning is not restricted by specific shapes, and detection robustness can be improved. Extensive experiments on four benchmarks, including the ICDAR2015, MSRA-TD500, CTW1500 and Totaltext, show that the proposed JMNET achieves competitive performance in terms of both accuracy and speed over the state-of-the-art methods. Particularly, our method can reach a remarkable F-measure of 85.5% at 52 FPS on Totaltext. Code is available at: https://github.com/sakura-910/JMNET. [ABSTRACT FROM AUTHOR]

This paper proposes a fast mode decision algorithm for 3D High Efficiency Video Coding (3D-HEVC) depth intra coding. In the current 3D-HEVC design, it is observed that for most of the cases, full rate-distortion (RD) cost search of Bi-Partition mode could be skipped since most coding units (CUs) of depth map are very flat or smooth while Bi-Partition modes are designed for CUs with edge or sharp transition. Using the rough RD cost value calculated by HEVC Rough Mode Decision as a selection threshold, we propose a fast Bi-Partition modes selection algorithm to speed up the encoding process. The test result for the proposed fast algorithm reports a 34.4% encoding time saving with a 0.3% bitrate increase on synthesized views for the All-Intra test case and negligible impact under the random access test case. Moreover, by simply varying the selection threshold, we can make a tradeoff between encoding time saving and bitrate loss based on the requirement of different applications. [ABSTRACT FROM AUTHOR]