Search

Light field datasets enable researchers to conduct both objective and subjective quality assessments, which are particularly useful when acquisition equipment or resources are not available. Such datasets may vary in terms of capture technology and methodology, content, quality characteristics (e.g., resolution), and the availability of subjective ratings. When contents of a light field dataset are visualized on a light field display, the display system matches the received input to its output capabilities through various processes, such as interpolation. Therefore, one of the most straightforward methods to create light field contents for a specific display is to consider its visualization parameters during acquisition. In this paper, we introduce a novel display-specific light field dataset, captured using a DSLR camera and a turntable rig. The visual data of the seven static scenes were recorded twice by using two settings of angular resolution. While both were acquired uniformly within a 53-degree angle, which matches the viewing cone of the display they were captured for, one dataset consists of 70 views per content, while the other of 140. Capturing the contents twice was a more straightforward solution than downsampling, as the latter approach could either degrade the quality or make the FOV size inaccurate. The paper provides a detailed characterization of the captured contents, as well as compressed variations of the contents with various codecs, together with the calculated values of commonly-used quality metrics for the compressed light field contents. We expect that this dataset will be useful for the research community working on light field compression, processing, and quality assessment, for instance to perform subjective quality assessment tests on a display with a 53-degree display cone and to test new interpolation methods and objective quality metrics. In future work, we will also focus on subjective tests and provide relevant results. This dataset is made free to access for the research community. [ABSTRACT FROM AUTHOR]

Over the years, there has been growing interest in multimedia services, especially in the video domain, where firms and subscribers require higher resolutions, framerates, and sampling precision. This results in a huge amount of data that needs to be processed, stored, and transmitted. As a result, researchers face the challenge of developing new compression standards that can reduce the amount of data while maintaining the same quality. In this paper, the compression performance of the latest and most commonly used video codecs, namely H.266/VVC, AV1, H265/HEVC, and H.264/AVC was examined. The test set included seven sequences of various content at 8K, Ultra HD (UHD), and Full HD (FHD) resolutions, encoded to bitrates ranging from 1 to 15 Mbps for FHD and UHD resolutions and from 5 to 50 Mbps for 8K resolution. Objective quality metrics, such as peak signal-to-noise ratio (PSNR), the structural similarity index (SSIM), and video multi-method assessment fusion (VMAF) were used to measure codec performance. The results showed that H.266/VVC outperformed all other codecs, namely H.264/AVC, H.265/HEVC, and AV1, in terms of the Bjøntegaard delta (BD) model. The average bitrate savings were approximately 78% for H.266/VVC, 63% for AV1, and 53% for H.265/HEVC relative to H.264/AVC, 59% for H.266/VVC and 22% for AV1 compared to H.264/AVC, and 46% for H.266/VVC relative to AV1 (all for 8K resolution). The results also showed that codec performance varied depending on resolution, with higher resolutions showing greater efficiency for newly developed codecs, such as H.266/VVC and AV1. This confirms the fact that the H.266/VVC and AV1 codecs were primarily developed for videos at high resolutions, such as 8K and/or UHD. [ABSTRACT FROM AUTHOR]

This paper is dedicated to video coding based on a compressive sensing (CS) framework. In CS, it is assumed that if a video sequence is sparse in some transform domain, then it could be reconstructed from a much lower number of samples (called measurements) than the Nyquist–Shannon theorem requires. Here, the performance of such a codec depends on how the measurements are acquired (or sensed) and compressed and how the video is reconstructed from the decoded measurements. Here, such a codec potentially could provide significantly faster encoding compared with traditional block-based intra-frame encoding via Motion JPEG (MJPEG), H.264/AVC or H.265/HEVC standards. However, existing video codecs based on CS are inferior to the traditional codecs in rate distortion performance, which makes them useless in practical scenarios. In this paper, we present a video codec based on CS called CS-JPEG. To the author's knowledge, CS-JPEG is the first codec based on CS, combining fast encoding and high rate distortion results. Our performance evaluation shows that, compared with the optimized software implementations of MJPEG, H.264/AVC, and H.265/HEVC, the proposed CS-JPEG encoding is 2.2, 1.9, and 30.5 times faster, providing 2.33, 0.79, and 1.45 dB improvements in the peak signal-to-noise ratio, respectively. Therefore, it could be more attractive for video applications having critical limitations in computational resources or a battery lifetime of an upstreaming device. [ABSTRACT FROM AUTHOR]

High-Efficiency Video Coding (HEVC) is one of the most widely studied coding standards. It still uses the block-based hybrid coding framework of Advanced Video Coding (AVC), and compared to AVC, it can double the compression ratio while maintaining the same quality of reconstructed video. The quantization module is an important module in video coding. In the process of quantization, quantization parameter is an important factor in determining the bitrate in video coding, especially in the case of limited channel bandwidth. It is particularly important to select a reasonable quantization parameter to make the bitrate as close as possible to the target bitrate. Aiming at the problem of unreasonable selection of quantization parameters in codecs, this paper proposes using a differential evolution algorithm to assign quantization parameter values to the coding tree unit (CTU) in each frame of 360-degree panoramic video based on HEVC so as to strike a balance between bitrate and distortion. Firstly, the number of CTU rows in a 360-degree panoramic video frame is considered as the dimension of the optimization problem. Then, a trial vector is obtained by randomly selecting the vectors in the population for mutation and crossover. In the mutation step, the algorithm generates a new parameter vector by adding the weighted difference between two population vectors to a third vector. And the elements in the new parameter vector are selected according to the crossover rate. Finally, the trial vector is regarded as the quantization parameter of each CTU in the CTU row to encode, and the vector with the least rate distortion is selected. The algorithm will produce the optimal quantization parameter combination for the current video. The experimental results show that compared to the benchmark algorithm of HEVC reference software HM-16.20, the proposed algorithm can provide a bit saving of 1.86%, while the peak signal-to-noise ratio (PSNR) can be improved by 0.07 dB. [ABSTRACT FROM AUTHOR]

Existing deep learning architectures usually use a separate encoder and decoder to generate the desired simulated images, which is inefficient for feature analysis and synthesis. Aiming at the problem that the existing methods fail to fully utilize the correlation of codecs, this paper focuses on the codec-unified invertible networks to accurately guide the image deblurring process by controlling latent variables. Inspired by U-Net, a U-shaped multi-level invertible network (UML-IN) is proposed by integrating the wavelet invertible networks into a supervised U-shape architecture to establish the multi-resolution correlation between blurry and sharp image features under the guidance of hybrid loss. Further, this paper proposes to use L 1 regularization constraints to obtain sparse latent variables, thereby alleviating the information dispersion problem caused by high-dimensional inference in invertible networks. Finally, we fine-tune the weights of invertible modules by calculating a similarity loss between blur-sharp variable pairs. Extensive experiments on real and synthetic blurry sets show that the proposed approach is efficient and competitive compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The new event cameras are now widely used in many computer vision applications. Their high raw data bitrate levels require a more efficient fixed-length representation for low-bandwidth transmission from the event sensor to the processing chip. A novel low-complexity lossless compression framework is proposed for encoding the synchronous event frames (EFs) by introducing a novel memory-efficient fixed-length representation suitable for hardware implementation in the very-low-power (VLP) event-processing chip. A first contribution proposes an improved representation of the ternary frames using pixel-group frame partitioning and symbol remapping. Another contribution proposes a novel low-complexity memory-efficient fixed-length representation using multi-level lookup tables (LUTs). Complex experimental analysis is performed using a set of group-size configurations. For very-large group-size configurations, an improved representation is proposed using a mask-LUT structure. The experimental evaluation on a public dataset demonstrates that the proposed fixed-length coding framework provides at least two times the compression ratio relative to the raw EF representation and a close performance compared with variable-length video coding standards and variable-length state-of-the-art image codecs for lossless compression of ternary EFs generated at frequencies bellow one KHz. To our knowledge, the paper is the first to introduce a low-complexity memory-efficient fixed-length representation for lossless compression of synchronous EFs, suitable for integration into a VLP event-processing chip. [ABSTRACT FROM AUTHOR]

IP-PBX have grown considerably in the telecoms and IT industries in recent times due to their low cost, high customizability, open source and hardware-independent nature. Unlike traditional PBX systems that are TDM-based and require a separate wired infrastructure to operate, an IP PBX can be attached to the local LAN and uses SIP with RTP over Internet Protocol (IP) for signaling and data transport, respectively. Asterisk-based IP PBX has become a de facto standard for open-source low-to-medium calling capacity requirements. Single Board Computers (SBCs) are attracting considerable attention from academics, hobbyists and the industry, due to their compact size, low cost, low power, portability and fair processing ability. Raspberry Pi series SBCs are the pioneer in the field, having Pi, Pi2, Pi3 and Pi Zero in production, supporting different processing and storage capabilities. An IP PBX embedded on the raspberry Pi SBC can serve an adequate number of users and provide a tiny platform that can fit in a pocket and be transported anywhere while preserving the functionality of a complete IP PBX. Such systems can be phenomenally successful in situations where large infrastructures cannot be taken, for instance, in warzones or disaster-affected areas where communication networks are damaged, necessitating an immediate requirement for an ad hoc-based communication system. In this paper, we aim to explore the performance of an Asterisk®-based IP PBX installed on a Raspberry Pi 3 (model B) platform in terms of concurrent call handling using different codecs. [ABSTRACT FROM AUTHOR]

With the advancement in next-generation communication technologies, the so-called Tactile Internet is getting more attention due to its smart applications, such as haptic-enabled teleoperation systems. The stringent requirements such as delay, jitter, and packet loss of these delay-sensitive and loss-intolerant applications make it more challenging to ensure the Quality of Service (QoS) and Quality of Experience (QoE). In this regard, different haptic codec and control schemes were proposed for QoS and QoE provisioning in the Tactile Internet. However, they maximize the QoE while degrading the system's stability under varying delays and high packet rates. In this paper, we present a reinforcement learning-based Intelligent Tactile Edge (ITE) framework to ensure both transparency and stability of teleoperation systems with high packet rates and variable time delay communication networks. The proposed ITE first estimates the network challenges, including communication delay, jitter, and packet loss, and then utilizes a Q-learning algorithm to select the optimal haptic codec scheme to reduce network load. The proposed framework aims to explore the optimal relationship between QoS and QoE parameters and make the tradeoff between stability and transparency during teleoperations. The simulation result indicates that the proposed strategy chooses the optimal scheme under different network impairments corresponding to the congestion level in the communication network while improving the QoS and maximizing the QoE. The end-to-end performance of throughput (1.5 Mbps) and average RTT (70 ms) during haptic communication is achieved with a learning rate and discounted factor value of 0.5 and 0.8, respectively. The results indicate that the communication system can successfully achieve the QoS and QoE requirements by employing the proposed ITE framework. [ABSTRACT FROM AUTHOR]

The surge in ultra-high-definition video content has intensified the demand for advanced video compression techniques. Video encoding preprocessing can improve coding efficiency while ensuring a high degree of compatibility with existing codecs. Existing video encoding preprocessing methods are limited in their ability to fully exploit redundant features in video data and recover high-frequency details, and their network architectures often lack compatibility with neural video encoders. To addressing these challenges, we propose a Multi-Dimensional Enhancement and Reconstruction (MDER) preprocessing method to improve the efficiency of deep learning-based neural video encoders. Firstly, our approach integrates a degradation compensation module to mitigate encoding noise and boost feature extraction efficiency. Secondly, a lightweight fully convolutional neural network is employed, which utilizes residual learning and knowledge distillation to refine and suppress irrelevant features across spatial and channel dimensions. Furthermore, to maximize the use of redundant information, we incorporate Dense Blocks, which can enhance and reconstruct important features in the video data during preprocessing. Finally, the preprocessed frames are then mapped from pixel space to feature space through the Dense Feature-Enhanced Video Compression (DFVC) module, which improves motion estimation and compensation accuracy. The experimental results demonstrate that, compared to neural video encoders, the MDER method can reduce bits per pixel (Bpp) by 0.0714 and 0.0536 under equivalent PSNR and MS-SSIM conditions, respectively. These results demonstrate significant improvements in compression efficiency and reconstruction quality, highlighting the effectiveness of the MDER preprocessing method and its compatibility with neural video codec workflows. [ABSTRACT FROM AUTHOR]

The surge in image data has significantly increased the pressure on storage and transmission, posing new challenges for image compression technology. The structural texture of an image implies its statistical characteristics, which is effective for image encoding and decoding. Consequently, content-adaptive compression methods based on learning can better capture the content attributes of images, thereby enhancing encoding performance. However, learned image compression methods do not comprehensively account for both the global and local correlations among the pixels within an image. Moreover, they are constrained by rate-distortion optimization, which prevents the attainment of a compact representation of image attributes. To address these issues, we propose a syntax-guided content-adaptive transform framework that efficiently captures image attributes and enhances encoding efficiency. Firstly, we propose a syntax-refined side information module that fully leverages syntax and side information to guide the adaptive transformation of image attributes. Moreover, to more thoroughly exploit the global and local correlations in image space, we designed global–local modules, local–global modules, and upsampling/downsampling modules in codecs, further eliminating local and global redundancies. The experimental findings indicate that our proposed syntax-guided content-adaptive image compression model successfully adapts to the diverse complexities of different images, which enhances the efficiency of image compression. Concurrently, the method proposed has demonstrated outstanding performance across three benchmark datasets. [ABSTRACT FROM AUTHOR]

The growing demand for high-quality video transmission over bandwidth-constrained networks and the increasing availability of video content have led to the need for efficient storage and distribution of large video files. To improve the latter, this article offers a comparison of six video compression methods without loss of quality. Particularly, H.255, VP9, AV1, convolutional neural network (CNN), recurrent neural network (RNN), and deep autoencoder (DAE). The proposed decision is to use a dataset of high-quality videos to implement and compare the performance of classical compression algorithms and algorithms based on machine learning. Evaluations of the compression efficiency and the quality of the received images were made on the basis of two metrics: PSNR and SSIM. This comparison revealed the strengths and weaknesses of each approach and provided insights into how machine learning algorithms can be optimized in future research. In general, it contributed to the development of more efficient and effective video compression algorithms that can be useful for a wide range of applications. [ABSTRACT FROM AUTHOR]

In this paper, we propose a partial decoding method with limited memory usage for high-speed thumbnail extraction. The proposed method performs a partial inverse transform and a partial intra prediction in order to reconstruct pixels for intra prediction and thumbnails. Thereafter, the reconstructed pixels at the bottom and right line of the block are stored in the line buffer and the thumbnail buffer without being stored in the decoded picture buffer with full resolution. H.264/AVC, HEVC and VP9 video codecs have different coding structures, prediction and transforms; however, the proposed algorithm can be applied to the corresponding codecs in the same manner. In order to evaluate the performance of the proposed method, we implemented the proposed algorithm for H.264/AVC, HEVC and VP9. We found that the thumbnail extraction time of the proposed method decreased by 66% in H.264/AVC, 52% in HEVC and 48% in VP9 as compared to the full decoding method. [ABSTRACT FROM AUTHOR]

It is challenging to find a proper way to compress computer-generated holography (CGH) data owing to their huge data requirements and characteristics. This study proposes CGH data coding systems with high-efficiency video coding (HEVC), three-dimensional extensions of HEVC (3D-HEVC), and video-based point cloud compression (V-PCC) codecs. In the proposed system, we implemented a procedure for codec usage and format conversion and evaluated the objective and subjective results to analyze the performance of the three coding systems. We discuss the relative advantages and disadvantages of the three coding systems with respect to their coding efficiency and reconstruction results. Our analysis concluded that 3D-HEVC and V-PCC are potential solutions for compressing red, green, blue, and depth (RGBD)-sourced CGH data. [ABSTRACT FROM AUTHOR]

Finding a proper balance between video quality and the required bandwidth is an important issue, especially in networks of limited capacity. The problem of comparing the efficiency of video codecs and choosing the most suitable one in a specific situation has become very important. This paper proposes a method of comparing video codecs while also taking into account objective quality assessment metrics. The author shows the process of preparing video footage, assessing its quality, determining the rate–distortion curves, and calculating the bitrate saving for pairs of examined codecs. Thanks to the use of the spline interpolation method, the obtained results are better than those previously presented in the literature, and more resistant to the quality metric used. [ABSTRACT FROM AUTHOR]

As video compression is one of the core technologies that enables seamless media streaming within the available network bandwidth, it is crucial to employ media codecs to support powerful coding performance and higher visual quality. Versatile Video Coding (VVC) is the latest video coding standard developed by the Joint Video Experts Team (JVET) that can compress original data hundreds of times in the image or video; the latest audio coding standard, Unified Speech and Audio Coding (USAC), achieves a compression rate of about 20 times for audio or speech data. In this paper, we propose a pre-processing method to generate a two-dimensional (2D) audio signal as an input of a VVC encoder, and investigate the applicability to 2D audio compression using the video coding scheme. To evaluate the coding performance, we measure both signal-to-noise ratio (SNR) and bits per sample (bps). The experimental result shows the possibility of researching 2D audio encoding using video coding schemes. [ABSTRACT FROM AUTHOR]

The Joint Video Exploration Team (JVET) has created the Versatile Video Coding Standard (VVC/H.266), the most up-to-date video coding standard, offering a broad selection of coding tools. The maturity of commercial VVC codecs can significantly reduce costs and improve coding efficiency. However, the latest video coding standards have introduced binomial and trinomial tree partitioning methods, which cause the coding units (CUs) to have various shapes, increasing the complexity of coding. This article proposes a technique to simplify VVC intra prediction through the use of gradient analysis and a multi-feature fusion CNN. The gradient of CUs is computed by employing the Sobel operator, the calculation results are used for predecision-making. Further decisions can be made by CNN for coding units that cannot be judged whether they should be segmented or not. We calculate the standard deviation (SD) and the initial depth as the input features of the CNN. To implement this method, the initial depth can be determined by constructing a segmented depth prediction dictionary. For the initial segmentation depth of the coding unit, regardless of its shape, it can also be determined by consulting the dictionary. The algorithm can determine whether to split CUs of varying sizes, decreasing the complexity of the CU division process and making VVC more practical. Experimental results demonstrate that the proposed algorithm can reduce encoding time by 36.56% with a minimal increase of 1.06% Bjøntegaard delta bit rate (BD-BR) compared to the original algorithm. [ABSTRACT FROM AUTHOR]

This paper addresses the viral infectious watermarking (VIW) model using biological virus infection for a new-paradigm of video copyright protection of MPEG/H.264/AVC/HEVC. Our model aims to spread or infect the watermark to different codecs each time video contents are copied, edited, or transcoded. Thus, we regard the watermark as the infectious virus, the video content as the host, and the video codec as the contagion medium and then model pathogen, mutant, and contagion as the infectious watermark. Then, we define the techniques of viral infectious watermark generation, kernel-based VIW, and content-based VIW. Furthermore, we present a reversible VIW for fast infection in VIW model. This makes the video quality and strength be adaptively controlled in the infectious process. Experiment results verified that our VIW model can detect or recover the reversible watermark without loss in different codecs and also can maintain the quality of video content that is recovered to the same bit rate. [ABSTRACT FROM AUTHOR]

Adaptive multi-rate wideband (AMR-WB) speech codecs have been widely used for high speech quality in modern mobile communication systems, e.g., handheld mobile devices. Nevertheless, a major handicap is that a remarkable computational load is required in the vector quantization (VQ) of immittance spectral frequency (ISF) coefficients of an AMR-WB coding. In view of this, a two-stage search algorithm is presented in this paper as an efficient way to reduce the computational complexity of ISF quantization in AMR-WB coding. At stage 1, an input vector is assigned to a search subspace in an efficient manner using the binary search space-structured VQ (BSS-VQ) algorithm, and a codebook search is performed over the subspace at stage 2 using the iterative triangular inequality elimination (ITIE) approach. Through the use of the codeword rejection mechanisms equipped in both stages, the computational load can be remarkably reduced. As compared with the original version of the BSS-VQ algorithm, the upgraded version provides a computational load reduction of up to 51%. Furthermore, this work is expected to satisfy the energy saving requirement when implemented on an AMR-WB codec of mobile devices. [ABSTRACT FROM AUTHOR]

Adaptive streaming has become a key technology for various multimedia services, such as online learning, mobile streaming, Internet TV, etc. However, because of throughput fluctuations, video quality may be dramatically varying during a streaming session. In addition, stalling events may occur when segments do not reach the user device before their playback deadlines. It is well-known that quality variations and stalling events cause negative impacts on Quality of Experience (QoE). Therefore, a main challenge in adaptive streaming is how to evaluate the QoE of streaming sessions taking into account the influences of these factors. Thus far, many models have been proposed to tackle this issue. In addition, a lot of QoE databases have been publicly available. However, there have been no extensive evaluations of existing models using various databases. To fill this gap, in this study, we conduct an extensive evaluation of thirteen models on twelve databases with different characteristics of viewing devices, codecs, and session durations. Through experiment results, important findings are provided with regard to QoE prediction of streaming sessions. In addition, some suggestions on the effective employment of QoE models are presented. The findings and suggestions are expected to be useful for researchers and service providers to make QoE assessments and improvements of streaming solutions in adaptive streaming. [ABSTRACT FROM AUTHOR]

Since lossless compression can only achieve two to four times data compression, it may not be efficient to deploy lossless compression in bandwidth constrained applications. Instead, it would be more economical to adopt perceptually lossless compression, which can attain ten times or more compression without loss of important information. Consequently, one can transmit more images over bandwidth limited channels. In this research, we first aimed to compare and select the best compression algorithm in the literature to achieve a compression ratio of 0.1 and 40 dBs or more in terms of a performance metric known as human visual system model (HVSm) for maritime and sonar images. Our second objective was to demonstrate error concealment algorithms that can handle corrupted pixels due to transmission errors in interference-prone communication channels. Using four state-of-the-art codecs, we demonstrated that perceptually lossless compression can be achieved for realistic maritime and sonar images. At the same time, we also selected the best codec for this purpose using four performance metrics. Finally, error concealment was demonstrated to be useful in recovering lost pixels due to transmission errors. [ABSTRACT FROM AUTHOR]

We write the mutual information between an input speech utterance and its reconstruction by a code-excited linear prediction (CELP) codec in terms of the mutual information between the input speech and the contributions due to the short-term predictor, the adaptive codebook, and the fixed codebook. We then show that a recently introduced quantity, the log ratio of entropy powers, can be used to estimate these mutual informations in terms of bits/sample. A key result is that for many common distributions and for Gaussian autoregressive processes, the entropy powers in the ratio can be replaced by the corresponding minimum mean squared errors. We provide examples of estimating CELP codec performance using the new results and compare these to the performance of the adaptive multirate (AMR) codec and other CELP codecs. Similar to rate distortion theory, this method only needs the input source model and the appropriate distortion measure. [ABSTRACT FROM AUTHOR]