Search

The latest video coding standard, versatile video coding (VVC), achieves almost twice coding efficiency compared to its predecessor, the high efficiency video coding (HEVC). However, achieving this efficiency (for intra coding) requires 31 × computational complexity compared to HEVC, which makes it challenging for low power and real-time applications. This paper, proposes a novel machine learning approach that jointly and separately employs two modalities of features, to simplify the intra coding decision. To do so, first a set of features are extracted that use the existing DCT core of VVC, to assess the texture characteristics, and forms the first modality of data. This produces high-quality features with almost no extra computational overhead. The distribution of intra modes at the neighboring blocks is also used to form the second modality of data, which provides statistical information about the frame, unlike the first modality. Second, a two-step feature reduction method is designed that reduces the size of feature set, such that a lightweight model with a limited number of parameters can be used to learn the intra mode decision task. Third, three separate training strategies are proposed (1) an offline training strategy using the first (single) modality of data, (2) an online training strategy that uses the second (single) modality, and (3) a mixed online–offline strategy that uses bimodal learning. Finally, a low-complexity encoding algorithms is proposed based on the proposed learning strategies. Extensive experimental results show that the proposed methods can reduce up to 24% of encoding time, with a negligible loss of coding efficiency. Moreover, it is demonstrated how a bimodal learning strategy can boost the performance of learning. Lastly, the proposed method has a very low computational overhead (0.2%), and uses existing components of a VVC encoder, which makes it much more practical compared to competing solutions. [ABSTRACT FROM AUTHOR]

In Hybrid Cloud Gaming (HCG), as long as the graphical assets are available at the client side, rendering can be performed locally. However, if the client is not able to render all the frames in time, this may result in the frame rate to drop under the target value. This paper presents an Asset-based frame-level hybrid graphics/video rate control method for HCG, referred to as AHCG, which aims at improving the Quality of Experience (QoE) of players by tapping into the available processing power at the client side, while keeping a steady frame rate for thin clients and taking full advantage of the user's tolerable delay. In the proposed client/server model, graphics data are intercepted and streamed in an asset-based approach. This approach handles the rate control issue per asset which is defined to be 3D object model data, textures, and shader programs. Rendering a frame on the client side not only maintains original quality for that frame, but it also reduces bandwidth requirements of the entire service by reusing the same assets for different frames. In the proposed method, if the accumulated rendering delay at the client side violates the tolerable delay set by the user, video streaming is used to compensate for the client device's lack of processing power. Furthermore, quality fluctuation between graphics and video frames is addressed to provide a seamless experience when switching between graphics and video streaming. Several objective and subjective tests are conducted and the experimental results show a 20-fps increase in frame rate while maintaining a minimum value of 58 fps, with a minimum of 0.25 units improvement in the pooled standard deviation of SSIM values, compared to existing HCGs. Also, the subjective tests suggest an average 5.62 percent improvement in MOS compared to best HCG methods. [ABSTRACT FROM AUTHOR]

The high-efficiency video coding (HEVC) standard improves the coding efficiency at the cost of a significantly more complex encoding process. This is an issue for a large number of video-capable devices that operate on batteries, with limited and varying processing power. A complexity controller enables an encoder to provide the best possible quality at any power quota. This paper proposes a complexity control method for HEVC intra coding, based on a Pareto-efficient rate–distortion–complexity (R–D–C) analysis. The proposed method limits the intra prediction for each block (as opposed to existing methods which limit the block partitioning), on a frame-level basis. This method consists of three steps, namely rate-complexity modeling, complexity allocation, and configuration selection. In the first step, a rate-complexity model is presented which estimates the encoding complexity according to the compression intensity. Then, according to the estimated complexity and target complexity, a complexity budget is allocated to each frame. Finally, an encoding configuration from a set of Pareto-efficient configurations is selected according to the allocated complexity and the video content, which offers the best compression performance. Experimental results indicate that the proposed method can adjust the complexity from 100 to 50%, with a mean error rate of less than 0.1%. The proposed method outperforms many state-of-the-art approaches, in terms of both control accuracy and compression efficiency. The encoding performance loss in terms of BD-rate varies from 0.06 to 3.69%, on average, for 90–60% computational complexity, respectively. The method can also be used for lower than 50% complexity if need be, with a higher BD-rate. [ABSTRACT FROM AUTHOR]

Motion Estimation (ME) is one of the most computationally demanding parts of video encoders. The Test Zone (TZ) search is a popular fast ME algorithm, which is recommended for High-Efficiency Video Coding (HEVC). While the TZ search achieves an excellent coding efficiency, it is not a favorable choice for hardware implementations due to 1) a relatively high computational complexity, 2) inducing data dependency among the neighboring blocks, which complicates hardware implementations and parallel processing in software implementations, and 3) lack of computational adjustability, which is required for video encoding in power-constrained devices. This paper diagnoses the cause of these issues to be in the multiple starting search points of the TZ search algorithm. Accordingly, a method is proposed to find a single reliable starting point that replaces the first step of the TZ search algorithm. To do so, both current and reference frames are analyzed using a complex wavelet transform, and similar salient points are identified among the two frames. Then a light-weight process is used to match these points to find a single reliable starting point. The reliability of this point leads to reduced zonal refinement range with negligible cost in compression efficiency. Since adjusting the refinement range can be used as an effective way for adjusting the complexity, this results in a computationally scalable ME algorithm, named FMECWT. In contrast to the existing methods, FMECWT does not rely on neighboring blocks, which eliminates the inherent data dependency of TZ search. Experimental results show that FMECWT achieves ~35% to ~85% ME time reduction compared to TZ search, with only 0.1% to 1.7% increase in BD-Rate. [ABSTRACT FROM AUTHOR]

Grounded theory (GT), a widespread qualitative research methodology, has been introduced to generate a theory from data that is gathered from one discipline. However, in many studies, we are required to involve multiple disciplines and generate a theory that explains a multidisciplinary phenomenon and incorporate combined concepts. The combined concept is either a generalized concept from one of the disciplines or a new abstract concept that can satisfy the properties of multiple disciplines. The GT coding phases are not able to extract these combined concepts. In this paper, an extended GT has been proposed. The proposed method can operate over multiple disciplines and extract their corresponding combined concepts. The proposed extended grounded theory (EGT) uses ontology matching and merging methods to merge the extracted concepts and proposes two new coding phases; namely "Description" and "Combination" phases. These phases overcome the challenge of describing combined concepts, the large number of concept dependencies, and the propagation of the description changes from one concept to other concepts. As a use case, the proposed methodology has been implemented over two disciplines of "Web Security" and "Computation over Encrypted Data" and resulted in the unified theory of "Developing Secure Web Applications over Encrypted Data." This unified theory includes combined concepts such as Web Access Control over encrypted data and Operation of middle-boxes over encrypted data. In addition, EGT has been able to discover new combined concepts that were undefined previously in each of the two disciplines (e.g. XSS prevention with searchable encryption). As a second case study, EGT has been implemented over two disciplines of "Healthcare" and "Data Science" to generate a unified theory that explains the multidisciplinary phenomenon of "Privacy Preservation of Cloud based Health Data". In addition to a unified theory, EGT generates a conceptual framework of the combined and new concepts that can be used for requirement elicitation and analysis. Furthermore, the researcher can use the concepts' relationships to suggest a set of best-fit solutions to the elicited requirements. [ABSTRACT FROM AUTHOR]

To address the vast variety of computing requirements in recent ubiquitous computing ecosystem, there is a constant need for more complex computing systems that consist of integrated hardware (HW) and software (SW) systems. Providing an architectural insight into such systems helps in achieving a more efficient usage of system resources, verifying the characteristics of a platform and provisioning of its security and trust. Architecture reconstruction (AR) has been used in software engineering to gain a deeper insight into specific software. Neither software AR nor hardware reverse engineering techniques are sufficient to extract the architecture of a system that incorporates both HW/SW, since they are unable to recover the relationships between the HW and SW components. Inspired by the Symphony software AR framework, we propose a method to reconstruct the architecture of a computing platform as a whole. In order to cover the wide variety of existing HW/SW technologies, our method uses an ontology-based approach. Due to the lack of a comprehensive ontology in literature, we developed PLATOnt, a new ontology that has been shown to be more effective by OntoQA evaluation framework. We used our AR method to reconstruct the architecture of an ARM-based trusted execution environment and a Raspberry Pi platform, widely used in embedded systems and IoT devices. [ABSTRACT FROM AUTHOR]

The High Efficiency Video Coding (HEVC) adopts 35 intra prediction modes to provide a more precise intra prediction. As a result, HEVC intra coding is much more complex. To provide a flexible solution for battery-powered devices, which include varying power budgets based on battery level, this paper jointly employs statistical distribution of different intra modes, and texture analysis. While the statistical information specifies the most efficient modes for extremely low power budgets, for moderate to high power budgets the best angular modes are predicted by analyzing the residual of the Planar mode, which is demonstrated to represent the dominant edges of the block. Moreover, a scheme is presented that further reduces the complexity by predicting the more important DC and Planar modes using the information provided by the entropy coder. The proposed method imposes almost no computational overhead since it utilizes the existing internal operators of HEVC. Ultimately, an intra prediction scheme with seven predefined levels of complexity is presented that provides coding for various power budgets and devices with various capabilities. The highest level of this scheme provides ~23% total encoding time reduction compared to HM, in all-intra configuration, with only 0.78% penalty on BD-Rate for high end devices. While the lowest level makes encoding possible for extremely low power devices and low battery plans with ~52% time reduction (almost the entire share of intra coding) at the cost of 9.6% BD-Rate, which is shown to be nearly minimal for this level of processing power. [ABSTRACT FROM AUTHOR]

The new video coding standard, High Efficiency Video Coding (HEVC) uses up to 33 angular intra modes to achieve better coding efficiency in intra coding. While larger number of modes results in a significant increase in coding efficiency than its predecessor H.264/AVC, it leads to much more complexity. To reduce this complexity, this paper proposes a novel method based on the dual-tree complex wavelet Transform (CWT). As CWT is effectively capable of representing edges in various directions, it is used to estimate the best intra angular mode in each coding block. The encoder then only analyzes a few modes around the best estimated mode. The experimental results show that searching only six modes around the estimated mode (estimated mode ± 3) plus DC and planar can predict the best mode in almost 80 % of the times. In fact the remaining unpredicted blocks have such unspecified behavior that even their random selections do not have any side effects. With all-intra configuration, searching with this method among the estimated mode ± 3 and DC and planar can improve the intra frame coding time by up to 4-10 times faster and can save the total encoding time over 32 %, while causing less than 0.4 % increase in bitrate and preserving the quality. [ABSTRACT FROM AUTHOR]

Mode decision has a significant effect on the quality and complexity of video coding. It is even more challenging when generating multiple bitstreams with different bitrates (BRs) in, for example, dynamic adaptive streaming for HTTP or transrating systems. Full search and simplified fast search mode decision (MD) methods either suffer from a high computational complexity or have a negative impact on quality. Furthermore, mode selection in conventional approaches strongly depends on the quantization parameter (QP). Hence, modes that have been selected for high BR compression may not be suitable for low BR when transrating a bitstream. In this paper, we propose a rate-distortion (R-D)-optimized criterion for fast MD algorithms. The proposed cost function, when adopted in different fast MD algorithms, not only improves the R-D performance by up to 6.6% in terms of Bjøntegaard delta rate, but also reduces the execution time of the encoder by up to 6.8%. We also show that modes selected by the proposed criterion are less sensitive to changes in BR or QP. As a result, the same modes in an encoded bitstream may be used even after transrating using requantization, resulting in a significant R-D performance improvement of up to 33.3%. [ABSTRACT FROM PUBLISHER]

Cache miss can have a major impact on overall performance of many‐core systems. A miss may result in extra traffic and delay because of coherency messages. This has been reduced in coarse‐grain coherency protocols where only shared misses require a coherency message. Conventional off‐chip methods manage the shared miss rate by relying on reuse histories. However the pertinent memory overhead that comes with reuse histories makes them impractical for on‐chip multi‐processor systems. In this study, a new scheme has been proposed to reduce shared cache miss rate in multi‐processor system‐on‐chips that benefits from novel prefetching techniques to L2 caches from off‐chip memories or other remote L2 caches located on‐chip. In the proposed scheme, the previously proposed Virtual Tree Coherence (VTC) method has been extended to limit block forwarding messages to true sharers within each region. Instead of relying on exact reuse histories, shared regions are searched for regional, temporal and statistical similarities. These similarities are exploited for determining the sharers that should receive the forwarded blocks. The proposed method has been evaluated with Splash‐2 workloads. Simulation results indicate that the proposed method has reduced shared miss count by up to 75%, and improved interconnect traffic by up to 47% compared with VTC. [ABSTRACT FROM AUTHOR]

With the growing demand for 3D video, efforts are underway to incorporate it in the next generation of broadcast and streaming applications and standards. 3D video is currently available in games, entertainment, education, security, and surveillance applications. A typical scenario for multiview 3D consists of several 3D video sequences captured simultaneously from the same scene with the help of multiple cameras from different positions and through different angles. Multiview video coding provides a compact representation of these multiple views by exploiting the large amount of inter-view statistical dependencies. One of the major challenges in this field is how to transmit the large amount of data of a multiview sequence over error prone channels to heterogeneous mobile devices with different bandwidth, resolution, and processing/battery power, while maintaining a high visual quality. Scalable Multiview 3D Video Coding (SMVC) is one of the methods to address this challenge; however, the evaluation of the overall visual quality of the resulting scaled-down video requires a new objective perceptual quality measure specifically designed for scalable multiview 3D video. Although several subjective and objective quality assessment methods have been proposed for multiview 3D sequences, no comparable attempt has been made for quality assessment of scalable multiview 3D video. In this article, we propose a new methodology to build suitable objective quality assessment metrics for different scalable modalities in multiview 3D video. Our proposed methodology considers the importance of each layer and its content as a quality of experience factor in the overall quality. Furthermore, in addition to the quality of each layer, the concept of disparity between layers (inter-layer disparity) and disparity between the units of each layer (intra-layer disparity) is considered as an effective feature to evaluate overall perceived quality more accurately. Simulation results indicate that by using this methodology, more efficient objective quality assessment metrics can be introduced for each multiview 3D video scalable modalities. [ABSTRACT FROM AUTHOR]

The motion JPEG2000 (MJP2) video coding standard uses intraframe compression and provides practical features such as fast coding, easy editing, and error robustness. Error robustness and concealment are some of the crucial requirements of any decoder in error prone applications. In this paper, a new error concealment method for MJP2 decoder has been proposed. The proposed method exploits the remaining temporal redundancies within the MJP2 coded video sequences for error concealment. The realized MJP2 decoder is able to detect and remove the artifacts produced by highly damaged bit planes that were not detected by the segmentation symbol, one of the error resilience tools in MJP2. Simulation results indicate that the proposed decoder can effectively remove such artifacts in the decoded erroneous MJP2 bit streams and improve the PSNR of the corrupted frames by up to 6.8 dB. An average of 0.2 dB improvement has been observed over a relatively large number of test cases and various error rates. [ABSTRACT FROM AUTHOR]