Search

The emerging immersive video coding standard moving picture experts group (MPEG) immersive video (MIV), which is ongoing standardization by MPEG-Immersive (MPEG-I) group, enables six degrees of freedom in a virtual reality environment that represents both natural and computer-generated scenes using multi-view video compression. The MIV eliminates the redundancy between multi-view videos and merges the residuals into multiple pictures, called an atlas. Thus, bitstreams with encoded atlases are generated and corresponding number of decoders are needed, which is challenging for the lightweight device with a single decoder. This article proposes a decoder-adaptive tiling and rate allocation method for MIV to overcome the challenge. First, the proposed method divides atlases into subpictures considering two aspects: (i) subpicture bitstream extracting and merging into one bitstream to use a single decoder and (ii) separation of each source view from the atlases for rate allocation. Second, the atlases are encoded by versatile video coding (VVC), using an extractable subpicture to divide the atlases into subpictures. Third, each subpicture bitstream is extracted, and asymmetric quality allocation for each subpictures is conducted by considering the residuals in the subpicture. Fourth, mixed-quality subpictures were merged by using the proposed bitstream merger. Fifth, the merged bitstream is decoded by using a single decoder. Finally, the viewing area of the user is synthesized by using the reconstructed atlases. Experimental results with the VVC test model (VTM) show that the proposed method achieves a 21.37% Bjøntegaard delta rate saving for immersive video peak signal-to-noise ratio and a 26.76% decoding runtime saving compared to the VTM anchor configuration. Moreover, it supports bitstreams for multiple decoders and single decoder without re-encoding, transcoding, or a substantial increase of the server-side storage. [ABSTRACT FROM AUTHOR]

The MPEG immersive video (MIV) has been designed to remove the redundancies of multi‐view videos and merge the residuals into two textures and two geometry videos; therefore, four bitstreams and video decoders are required to provide six degrees of freedom (6DoF) in virtual reality (VR) in MIV. This letter proposes a sub‐bitstream packing method and realisation in tiled streaming for MIV to provide a single bitstream for legacy systems that have a single decoder without an increase in the server‐side storage. The proposed method divided the input videos into tiles in the compressed‐domain to accomplish two goals: (i) generate a merged bitstream using the existing four bitstreams for legacy systems and (ii) improve the quality of an immersive video by disabling the deblocking filter. For a player with one or few numbers of decoders, the extracted tile sub‐streams were merged by the proposed four specific mixed‐quality tiling and packing methods using a motion‐constrained tile set (MCTS), while complying with the high‐efficiency video coding (HEVC) standard. Experimental results demonstrated that the proposed sub‐stream packing method required a single decoder and exhibited 17.40% decoding time and 0.81% bjontegaard‐delta rate (BD‐rate) savings on Y‐PSNR saving compared to the non‐tiled streaming method. [ABSTRACT FROM AUTHOR]

Biological regulatory pathways provide important information for target gene cancer therapy. Frequently, estimating the gene networks of two distinct patient groups is a worthwhile investigation. This paper proposes an approach, called jDAG, to the estimation of directed joint networks. It can identify common directed edges with joint data sets and distinct edges. In a simulation study, we show that the proposed jDAG outperforms existing methods although it does require longer computational times. We also present and discuss the example study of a breast cancer data set with ER + and ER-. [ABSTRACT FROM AUTHOR]

Numerous wearable technology companies have recently developed several head‐mounted display (HMD) products for virtual reality (VR) services. 5G wireless networks aim at providing high‐quality 3D multimedia services such as VR, augmented reality, and mixed reality. In this study, we examine the application of millimeter‐wave (mmWave) technology to realize low‐latency wireless communication between an HMD and its content server. However, the propagation characteristics of mmWave present several challenges such as the deafness, blockage, and beam alignment problems, and interference among content servers. In this study, we focus on an environment that provides VR services in the mmWave band and introduce existing techniques for addressing such challenges. In addition, we employ a commercialized IEEE 802.11ad VR dongle to measure the actual data rate of an mmWave VR application and identify the degree to which the performance deteriorates when the above problems occur. Finally, we verify the feasibility of the proposed solutions through a simulation of several VR scenarios in the mmWave band. [ABSTRACT FROM AUTHOR]

Visual impairments cause very limited and low vision, leading to difficulties in processing information such as obstacles, objects, multimedia contents (e.g., video, photographs, and paintings), and reading in outdoor and indoor environments. Therefore, there are assistive devices and aids for visually impaired (VI) people. In general, such devices provide guidance or some supportive information that can be used along with guide dogs, walking canes, and braille devices. However, these devices have functional limitations; for example, they cannot help in the processing of multimedia contents such as images and videos. Additionally, most of the available braille displays for the VI represent the text as a single line with several braille cells. Although these devices are sufficient to read and understand text, they have difficulty in converting multimedia contents or massive text contents to braille. This paper describes a methodology to effectively convert multimedia contents to braille using 2D braille display. Furthermore, this research also proposes the transformation of Digital Accessible Information SYstem (DAISY) and electronic publication (EPUB) formats into 2D braille display. In addition, it introduces interesting research considering efficient communication for the VI. Thus, this study proposes an eBook reader application for DAISY and EPUB formats, which can correctly render and display text, images, audios, and videos on a 2D multiarray braille display. This approach is expected to provide better braille service for the VI when implemented and verified in real-time. [ABSTRACT FROM AUTHOR]

In recent years, the rapid development of surveillance information in closed-circuit television (CCTV) has become an indispensable element in security systems. Several CCTV systems designed for video compression and encryption need to improve for the best performance and different security levels. Specially, the advent of 360 video makes the CCTV promising for surveillance without any blind areas. Compared to current systems, 360 CCTV requires the large bandwidth with low latency to run smoothly. Therefore, to improve the system performance, it needs to be more robust to run smoothly. Video transmission and transcoding is an essential process in converting codecs, changing bitrates or resizing the resolution for 360 videos. High-performance transcoding is one of the key factors of real time CCTV stream. Additionally, the security of video streams from cameras to endpoints is also an important priority in CCTV research. In this paper, a real-time transcoding system designed with the ARIA block cipher encryption algorithm is presented. Experimental results show that the proposed method achieved approximately 200% speedup compared to libx265 FFmpeg in transcoding task, and it could handle multiple transcoding sessions simultaneously at high performance for both live 360 CCTV system and existing 2D/3D CCTV system. [ABSTRACT FROM AUTHOR]

Market analysis reports suggest that the mobile virtual network operator (MVNO) market will experience subscriber growth at a compound annual growth rate (CAGR) of 10.7% from 2014 to 2020. This optimistic news presents great opportunities as well as many challenges. The major issue lies in the MVNOs' lack of spectrum allocation and an end-to-end network infrastructure, which means they must rely on mobile network operators (MNOs) for wireless access network connectivity to service their customers. In this article, we propose an approach that provides alternative wireless access methods to enable MVNOs to minimize their dependency on MNO networks. This leverages the fifth-generation (5G) softwarization technologies to provide selective multipath device-to-infrastructure (D2I) connections and service assortments. Supervised by software-defined networking (SDN)-based edge controllers, the multipath D2I connections provide flexible user data delivery over available communication resources without strict consideration of the MNO's infrastructure restraints. Additionally, a priority-based classification policy is applied to user data providing service assortments before it reaches the network infrastructure. Multiple pilot scenarios are implemented to verify the feasibility and performance tradeoff between the communication metrics of the proposed approach. [ABSTRACT FROM AUTHOR]

Abstract Today, the demand for and interest in virtual reality (VR) is increasing, since we can now easily experience VR in many applications. However, the computational ability of mobile VR is limited compared to that of tethered VR. Since VR represents a 360-degree area, providing high quality only for the area viewed by the user saves considerable bandwidth. Therefore, we propose a new tile-based streaming method that transforms 360-degree videos into mobile VR using high efficiency video coding (HEVC) and the scalability extension of HEVC (SHVC). While the SHVC base layer (BL) represents the entire picture, the enhancement layer (EL) can transmit only the desired tiles by applying the proposed method. By transmitting the BL and EL using region of interest (ROI) tiles, the proposed method helps reduce not only the computational complexity on the decoder side but also the network bandwidth. [ABSTRACT FROM AUTHOR]

This paper provides a novel real-time video streaming method in distributed wireless image-sensing platforms. It consists of (1) a millimeter-wave (mmW)-based multi-hop routing optimization for real-time video streaming, (2) wireless image-sensing platforms by using the high-efficiency video coding. A mmW wireless communication is a promising technology for increasing capacity in next-generation wireless systems. However, the weakness of mmW signals to (1) do long-distance transmission and (2) survive in non-line-of-sight environments makes the mmW networks need a multi-hop relaying. Thus, this paper focuses on the maximization of video transmission quality of service (QoS) that makes the optimization problem different from the conventional sum-rate maximization. Specifically, this paper develops an algorithm that optimizes the summation of QoS of the individual wireless transmission of different video streams, subject to constraints of separate streams (i.e., minimum requirements in each stream). Experimental results show the proposed approach presents 32 % better performance with 1.84 dB gain in Y-peak signal-to-noise ratio than the widely used max-min flow routing that is generally considered in QoS-sensitive video streaming applications. [ABSTRACT FROM AUTHOR]

Recently, the necessity of parallel ultra-high definition (UHD) video processing has been emerging, and the usage of the computing systems that have asymmetric processors, such as ARM big.LITTLE, is actively increasing. Thus, a new parallel UHD video processing method optimized for asymmetric multicore systems is essential. This paper proposes a novel High Efficiency Video Coding (HEVC) Tile partitioning method for the parallel processing by analyzing the computational power of asymmetric multicores. (1) The proposed method analyzes the computing power of asymmetric multicores and (2) the regression model of computational complexity per video resolution. Lastly, (3) the model determines the optimal HEVC Tile resolution for each core and partitions and allocates the Tiles to suitable cores. Experimental results with the test sequences of common test condition (CTC) show that the decoding speed improved by 17 % with implemented multi-threading module on ARM asymmetric multicore systems. [ABSTRACT FROM AUTHOR]

This paper presents a haptic telepresence system that enables visually impaired users to explore locations with rich visual observation such as art galleries and museums by using a telepresence robot, a RGB-D sensor (color and depth camera), and a haptic interface. The recent improvement on RGB-D sensors has enabled real-time access to 3D spatial information in the form of point clouds. However, the real-time representation of this data in the form of tangible haptic experience has not been challenged enough, especially in the case of telepresence for individuals with visual impairments. Thus, the proposed system addresses the real-time haptic exploration of remote 3D information through video encoding and real-time 3D haptic rendering of the remote real-world environment. This paper investigates two scenarios in haptic telepresence, i.e., mobile navigation and object exploration in a remote environment. Participants with and without visual impairments participated in our experiments based on the two scenarios, and the system performance was validated. In conclusion, the proposed framework provides a new methodology of haptic telepresence for individuals with visual impairments by providing an enhanced interactive experience where they can remotely access public places (art galleries and museums) with the aid of haptic modality and robotic telepresence. [ABSTRACT FROM PUBLISHER]

The delivery of high quality video to multiple viewers accessing the content from various devices and networks with varying conditions is a challenging task. Transferring the same content (often with large volume) to multiple viewers drives unicast transmission inefficient. Furthermore, traditional multicasting is not an optimal solution for heterogeneous networks since it either overwhelms slow receivers or starves the fast ones. Hence, new delivery solutions with the goal of reducing the output rate of the source and tailored to heterogeneous networks is necessary. In this paper, we present a novel scheme for layered video multicasting to viewers with the flexibility to access the content from various devices (in a heterogeneous network) and loss/erasure prone networks. In this scheme, the source node uses Scalable Video Coding (SVC) to obtain scalability together with modern elastic Forward Error Correction (FEC) for loss protection. There exists a non-trivial advantage in using elastic FEC in conjunction with SVC which enables us to introduce a new architecture that brings considerable savings in required source output bandwidth. Our solution adapts to the specific needs of viewers, shows resilience against packet loss, e.g. less than 5dB drop in Peak Signal-to-Noise Ratio (PSNR) for up to 30% loss rate, and especially, can reduce the source output rate by half for typical 3-layer SVC. [ABSTRACT FROM PUBLISHER]

The quality of video streaming in 38 GHz millimetre‐wave heterogeneous main‐cell and small‐cell networks in terms of signal‐to‐interference‐noise ratio is analysed. According to numerical simulation results, the impacts of interference from small‐cells on the quality of video streaming in main‐cells are numerically measured and analysed. [ABSTRACT FROM AUTHOR]

This paper presents two methodologies for delivering multimedia content to visually impaired people with the use of a haptic device and braille display. Based on our previous research, the research using Kinect v2 and haptic device with 2D+ (RGB frame with depth) data has the limitations of slower operational speed while reconstructing object details. Thus, this study focuses on the development of 2D multiarray braille display using an electronic book translator application because of its accuracy and high speed. This approach provides mobility and uses 2D multiarray braille display to represent media content contour more efficiently. In conclusion, this study achieves the representation of considerably massive text content compared to previous 1D braille displays. Besides, it also represents illustrations and figures to braille displays through quantization and binarization. [ABSTRACT FROM AUTHOR]

This paper describes an adaptive scheme of 360‐degree video streaming over millimeter‐wave (mmWave) communication for cyberphysical system. It consists of two parts, ie, (1) 360‐degree video streaming and offloading over mmWave 802.11ad 60 GHz wireless link and (2) 360‐degree video decoding, postprocessing, and display by using scalable high‐efficiency video coding. The mmWave communication is high‐speed wireless technology to increase the capacity of video transmission. However, current studies on video streaming over mmWave focus on long distance, and mobile devices process many tasks such as video decoding, postprocessing, and display by themselves. Therefore, the performance of mobile device seriously degrades in the case of high‐resolution video streaming, such as 4K. Furthermore, the synchronization of real‐time video streaming at high speed is an important issue to ensure the quality of service (QoS), especially when the mobile device is moving. Thus, this paper focuses on real‐time video streaming over mmWave with optimized QoS for high‐resolution video. Furthermore, this study implemented an offloading mechanism to handle the offloading tasks between a powerful GPU‐based PC and a mobile device. The experimental results show that the proposed method provides high‐performance 360‐degree video streaming in QoS‐sensitive video streaming applications. [ABSTRACT FROM AUTHOR]

360-degree video streaming for high-quality virtual reality (VR) is challenging for current wireless systems because of the huge bandwidth it requires. However, millimeter wave (mmWave) communications in the 60 GHz band has gained considerable interest from the industry and academia because it promises gigabit wireless connectivity in the huge unlicensed bandwidth (i.e., up to 7 GHz). This massive unlicensed bandwidth offers great potential for addressing the demand for 360-degree video streaming. This paper investigates the problem of 360-degree video streaming for mobile VR using the SHVC, the scalable of High-Efficiency Video Coding (HEVC) standard and PC offloading over 60 GHz networks. We present a conceptual architecture based on advanced tiled-SHVC and mmWave communications. This architecture comprises two main parts. (1) Tile-based SHVC for 360-degree video streaming and optimizing parallel decoding. (2) Personal Computer (PC) offloading mechanism for transmitting uncompressed video (viewport only). The experimental results show that our tiled extractor method reduces the bandwidth required for 360-degree video streaming by more than 47% and the tile partitioning mechanism was improved by up to 25% in terms of the decoding time. The PC offloading mechanism was also successful in offloading 360-degree decoded (or viewport only) video to mobile devices using mmWave communication and the proposed transmission schemes. [ABSTRACT FROM AUTHOR]

This paper describes the design and implementation of a multimedia home gateway for threescreen television (3STV) service. The proposed in-home wireless network uses scalable video coding (SVC) and unequal error protection with Raptor forward error correction (FEC) for maximizing the quality of experience (QoE) over the variable-bandwidth, error-prone wireless network. The gateway incorporates (a) dynamic SVC layerswitching, which enables the server to perform selecting appropriate layers from SVC bitstreams, (b) adaptive Raptor FEC, which controls the overhead of Raptor FEC according to packet loss rate (PLR), (c) an efficient combination of (a) and (b), and (d) slice group-based selective streaming in the overall gateway architecture. The paper explains the home gateway architecture as well as experiments for performance evaluation as compared to that of traditional SVC streaming. In the experiments conducted, gains in video quality vary from 2 to 5dB in peak signal-to-noise ratio (PSNR), with corresponding subjective improvements. Overall reductions of bit rate at the input to the home gateway vary from 28% to 36%1. [ABSTRACT FROM AUTHOR]

Recently, with the increasing demand for virtual reality (VR), experiencing immersive contents with VR has become easier. However, a tremendous amount of calculation and bandwidth is required when processing 360 videos. Moreover, additional information such as the depth of the video is required to enjoy stereoscopic 360 contents. Therefore, this paper proposes an efficient method of streaming high-quality 360 videos. To reduce the bandwidth when streaming and synthesizing the 3DoF+ 360 videos, which supports limited movements of the user, a proper down-sampling ratio and quantization parameter are offered from the analysis of the graph between bitrate and peak signal-to-noise ratio. High-efficiency video coding (HEVC) is used to encode and decode the 360 videos, and the view synthesizer produces the video of intermediate view, providing the user with an immersive experience.