Search

Modern video streaming services require quality assurance of the presented audiovisual material. Quality assurance mechanisms allow streaming platforms to provide quality levels that are considered sufficient to yield user satisfaction, with the least possible amount of data transferred. A variety of measures and approaches have been developed to control video quality, e.g., by adapting it to network conditions. These include objective matrices of the quality and thresholds identified by means of subjective perceptual judgments. The former group of matrices has recently gained the attention of (multi) media researchers. They call this area of study “Quality of Experience” (QoE). In this paper, we present a theoretical model based on review of previous QoE’s models. We argue that most of them represent the bottom-up approach to modeling. Such models focus on describing as many variables as possible, but with a limited ability to investigate the causal relationship between them; therefore, the applicability of the findings in practice is limited. To advance the field, we therefore propose a structural, top-down model of video QoE that describes causal relationships among variables. This novel top-down model serves as a practical guide for structuring QoE experiments, ensuring the incorporation of influential factors in a confirmatory manner.

This paper presents ‘Daily Video’, an innovative application designed to facilitate longitudinal studies in the realm of quality of experience (QoE). The application simulates real-life video service consumption and collects daily subjective quality assessments from users, promoting an in-depth understanding of user engagement and habits over time. Key features include scheduling, customised queries, user feedback, device orientation tracking and instant access to results, all of which enhance the application’s adaptability and usability for a variety of experimental conditions. The application, available as open-source code, supports state-of-the-art QoE research.

The paper presents different no reference (NR) objective metrics addressing the most important artefacts for raw (source) video sequences (noise, blur, exposure) and those introduced by compression (blocking, flickering) which can be used for assessing quality of experience. The validity of all metrics was verified under subjective tests.

Although longitudinal studies are common in other disciplines, such as psychology, medicine, or User Experience, they are rarely used in Quality of Experience (QoE). However, observing users over time can provide useful information on the QoE and help to better understand its influencing factors. Here, we present a systematic review of the methodologies of longitudinal studies in the QoE domain. We review papers selected through a systematic search and discuss various aspects of described studies, such as methods of gathering subjective assessment or length of the studies. Our work recognizes common practices that can be used to reproduce and extend the proposed long-term study designs. Additionally, based on our review, we propose future work directions for the longitudinal QoE studies.

Several video quality metrics (VQMs) have been proposed in many publications to predict how humans perceive video quality. It is common to observe significant disagreements amongst the quality predictions of these VQMs for the same video sequence. Following an extensive literature search, we found no publicised work that has investigated if such disagreements convey useful information on the accuracy of VQMs. Herein, a measure for quantifying the disagreement between VQMs is proposed. A small-scale subjective study is carried out to assess the effectiveness of our proposal. In particular, the proposed disagreement measure is shown to be extremely effective in determining whether the quality of any given processed video sequence (PVS) can be accurately predicted by the VQMs. This type of information is particularly useful for identifying video sequences that are likely to degrade the end-user’s quality of experience (QoE). Our proposal is also useful in selecting the most effective PVSs to be employed in a subjective test. We show that the proposed disagreement measure can be effectively predicted from bitstream features. This establishes a link between the capability to accurately assess the quality of a PVS and the way it is encoded. In addition, an analysis is conducted to compare the performances of some well-known and widely used open-source metrics and two proprietary metrics. The two proprietary metrics are used by a large media company for enhancing its delivery pipeline. The outcome of this comparison highlights the suitability of the open-source VQM, Video Multi-method Assessment Fusion (VMAF), as a good benchmark quality measure for both the industrial and academic environments.

We need data sets of images and subjective scores to develop robust no reference (or blind) visual quality metrics for consumer applications. These applications have many uncontrolled variables because the camera creates the original media and the impairment simultaneously. We do not fully understand how this impacts the integrity of our subjective data. We put forward two new data sets of images from consumer cameras. The first data set, CCRIQ2, uses a strict experiment design, more suitable for camera performance evaluation. The second data set, VIME1, uses a loose experiment design that resembles the behavior of consumer photographers. We gather subjective scores through a subjective experiment with 24 participants using the Absolute Category Rating method. We make these two new data sets available royalty-free on the Consumer Digital Video Library. We also present their integrity analysis (proposing one new approach) and explore the possibility of combining CCRIQ2 with its legacy counterpart. We conclude that the loose experiment design yields unreliable data, despite adhering to international recommendations. This suggests that the classical subjective study design may not be suitable for studies using consumer content. Finally, we show that Hoßfeld−Schatz−Egger α failed to detect important differences between the two data sets.

The key objective of no-reference (NR) visual metrics is to predict the end-user experience concerning remotely delivered video content. Rapidly increasing demand for easily accessible, high quality video material makes it crucial for service providers to test the user experience without the need for comparison with reference material. Nevertheless, the QoE measurement is not enough. The information about the source or error is very important as well. Therefore, the described system is based on calculating numerous dierent NR indicators, which are combined to provide the overall quality score. In this paper, more quality indicators than are used in the QoE calculation are described, since some of them detect specific errors. Such specific errors are dicult to include in a global QoE model but are important from the operation point of view.

The paper proposes a No-Reference (NR) quality assessment measurement originally developed for H.264, used for High Efficiency Video Coding (HEVC). In particular, authors present an investigation of NR metrics to objectively estimate the perceptual quality of a set of processed video sequences. The authors take into account typical distortions introduced by the block-based coding approaches like HEVC codec. The underlying processing used for the quality assessment considers the blockiness caused by the boundaries of each coded block and the blurring as a lack of spatial details. The correlation between the NR quality metrics and the well-known and most widely used objective metric, the Video Quality Model (VQM), is performed to validate the quality prediction accuracy based on the provided scores. The Pearson correlation coefficients obtained stand for promising results for different types of videos.