Search

Light field videos provide a rich representation of real-world, thus the research of this technology is of urgency and interest for both the scientific community and industries. Light field applications such as virtual reality and post-production in the movie industry require a large number of viewpoints of the captured scene to achieve an immersive experience, and this creates a significant burden on light field compression and streaming. In this paper, we first present a light field video dataset captured with a plenoptic camera. Then a new region-of-interest (ROI)-based video compression method is designed for light field videos. In order to further improve the compression performance, a novel view synthesis algorithm is presented to generate arbitrary viewpoints at the receiver. The experimental evaluation of four light field video sequences demonstrates that the proposed ROI-based compression method can save 5%-7% in bitrates in comparison to conventional light field video compression methods. Furthermore, the proposed view synthesis-based compression method not only can achieve a reduction of about 50% in bitrates against conventional compression methods, but the synthesized views can exhibit identical visual quality as their ground truth.

The utility of the horizon for airborne sense-and-avoid (ABSAA) applications is explored in this work. The horizon is a feature boundary across which an airborne scene can be separated into surface and sky and serves as a salient, heading-independent feature that may be mapped into an electro-optical sensor. The virtual horizon as established in this paper represents the horizon that would be seen assuming a featureless earth model and infinite visibility and is distinct from the apparent horizon in an imaging sensor or the pilot’s eye. For level flight, non-maneuvering collision course trajectories, it is expected that targets of interest will appear in close proximity to this virtual horizon. This paper presents a model for establishing the virtual horizon and its projection into a camera reference plane as part of the sensing element in an ABSAA system. Evaluation of the model was performed on a benchmark dataset of airborne collision geometries flown at the National Research Council (NRC) using the Cerberus camera array. The model was compared against ground truth flight test data collected using high accuracy inertial navigation systems aboard aircraft on several ’near-miss’ intercepts. The paper establishes the concept of ’virtual horizon proximity’ (VHP), the minimum distance from a detected target and the virtual horizon, and investigates the utility of using this metric as a means of rejecting false positive detections, and increasing range at first detection through the use of a region of interest (ROI) mask centred on the virtual horizon. The use of this horizon-centred ROI was shown to increase the range at first detection by an average factor of two, and was shown to reduce false positives for six popular feature detector algorithms applied across the suite of flight test imagery.

Region of interest (ROI) based image and video processing has attracted increased research efforts in recent years. The concept of perceptual ROI, however, is not always clearly defined leading to different interpretations between researchers related to bottom-up saliency (signal driven visual attention), top-down attention (subconscious, driven by higher cognitive factors, e.g. interest) or perceived interest. While all of these concepts are likely meaningful in the context of perceptual ROI based image and video processing, it is worth understanding how they are linked altogether. In this paper, the relationship between perceived interest and overt visual attention (which can cover both bottom-up and top-down attention) is studied. Towards this goal, a dedicated ROI selection experiment was performed and is analysed in detail, revealing deep insight into perceived interest in natural images. The outcomes are compared to an eye gaze tracking experiment representing overt visual attention in natural images. It is shown that there is indeed a strong relationship between perceived interest and overt visual attention for a wide range of natural scenes. We show that the relationship has a strong dependence on image content and presentation time during the eye gaze tracking experiment. Furthermore, eye gaze tracking data is revealed to have a high predictive value of primary ROI, particularly in case of the latter dominating over the remainder of the image. Both, the ROI and the eye gaze tracking databases are made publicly available to the research community. HighlightsStudy on the relationship between overt visual attention and perceived interest.Eye gaze tracking and ROI selection experiments as solid ground truths.Effects of image content and image presentation time are investigated in depth.Quantitative analysis shows strong relationship in images with dominant ROI.The data of both experiments is made publicly available to the research community.

International audience; Eye gaze is now used as a content adaptation trigger in interactive media applications, such as customized advertisement in video, and bit allocation in streaming video based on region-of-interest (ROI). The reaction time of a gaze-based networked system, however, is lower-bounded by the network round trip time (RTT). Furthermore, only low-sampling-rate gaze data is available when commonly available webcam is employed for gaze tracking. To realize responsive adaptation of media content even under non-negligible RTT and using common low-cost webcams, we propose a Hidden Markov Model (HMM) based gaze-prediction system that utilizes the visual saliency of the content being viewed. Specifically, our HMM has two states corresponding to two of human's intrinsic gaze behavioral movements, and its model parameters are derived offline via analysis of each video's visual saliency maps. Due to the strong prior of likely gaze locations offered by saliency information, accurate runtime gaze prediction is possible even under large RTT and using common webcam. We demonstrate the applicability of our low-cost gaze prediction system by focusing on ROI-based bit allocation for networked video streaming. To reduce transmission rate of a video stream without degrading viewer's perceived visual quality, we allocate more bits to encode the viewer's current spatial ROI, while devoting fewer bits in other spatial regions. The challenge lies in overcoming the delay between the time a viewer's ROI is detected by gaze tracking, to the time the effected video is encoded, delivered and displayed at the viewer's terminal. To this end, we use our proposed low-cost gaze prediction system to predict future eye gaze locations, so that optimized bit allocation can be performed for future frames. Through extensive subjective testing, we show that bit-rate can be reduced by up to 29\% without noticeable visual quality degradation when RTT is as high as 200 ms.

The viewing behaviour of human observers during image quality assessment is analysed. In this respect, the relative impact of image content and structural distortions is of particular interest. Two subjective experiments were conducted as a basis for analysis; a region-of-interest (ROI) experiment and an eye tracking experiment under quality assessment task. A correlation analysis and a receiver operating characteristics analysis reveal, that quality assessment takes place mainly within the ROI, indicating a higher impact of the content as compared to the distortions. This seems to be only contradicted by very strong and localised distortions outside the ROI.

Visual attention (VA) is an integral property of the human visual system. The deployment of a VA model can be beneficial for many image and video applications, such as, compression, transmission, and quality assessment. However, the design of a VA model is highly dependent on the subjective VA data used as ground truth and the application that the model is intended for. In this paper, we discuss two ways of obtaining subjective VA data that can subsequently be used to develop VA models; selective regions-of-interest and visual fixation patterns. The feasibility of both methods will be discussed, in particular, with respect to visual quality assessment.

Visual content typically exhibits regions that particularly attract the viewer's attention, usually referred to as regions-of-interest (ROI). In the context of visual quality one may expect that distortions occurring in the ROI are perceived more annoyingly than distortions in the background (BG). This is especially true given that the human visual system is highly space variant in sampling visual signals. However, this phenomenon of visual attention is only seldom taken into account in visual quality metric design. In this paper, we thus provide a framework for incorporation of visual attention into the design of an objective quality metric by means of region-based segmentation of the image. To support the metric design we conducted subjective experiments to both quantify the subjective quality of a set of distorted images and also to identify ROI in a set of reference images. Multiobjective optimization is then applied to find the optimal weighting of the ROI and BG quality metrics. It is shown that the ROI based metric design allows to increase quality prediction performance of the considered metric and also of two other contemporary quality metrics.

Three dimensional (3D) TV is becoming a mature technology due to the progress within areas such as display and network technology among others. However, 3D video demands a higher bandwidth in order to transmit the information needed to render or directly display several different views at the receiver. The 2D plus depth representation requires less bit rate than most 3D video representations, although the necessary views have to be rendered at the receiver. In this paper we propose to combine the 2D plus depth representation with region-of-interest (ROI) video coding to ensure a higher quality at parts of the sequence that are of interest to the viewer. These include objects close to the viewer as well as faces. This allows either the bit rate to be reduced by 12-28 % or the quality within the ROI to be increased by 0.57 - 1.5 dB, when a fixed bit rate is applied.

The state-of-the-art H.264 video codec is becoming widespread due to its excellent and advanced video coding properties. Due to band- with limitations, applications such as video surveillance, introduce the need for ecient mechanisms to visualize content-of-interest at maximal quality. Hence, support for region-of-interest functionality is of utmost importance. This paper will give an overview of available ROI solutions and will introduce a new method that uses an extensible ROI (xROI) to smooth the transition between the ROI and non-ROI part of the video. Moreover, several ROI shapes will be evaluated (e.g. diamond, rectangle, etc.). The extensive tests enable the formulation of a set recommenda- tions to reduce bandwidth consumption by making the proper choice of ROI features and their parameters (such as FMO type, ROI shape and quantization parameters).