Search

Structure-from-Motion (SfM) using the frames of a video sequence can be a challenging task because there is a lot of redundant information, the computational time increases quadratically with the number of frames, there would be low-quality images (e.g., blurred frames) that can decrease the final quality of the reconstruction, etc. To overcome all these issues, we present a novel deep-learning architecture that is meant for speeding up SfM by selecting frames using predicted sub-sampling frequency. This architecture is general and can learn/distill the knowledge of any algorithm for selecting frames from a video for generating high-quality reconstructions. One key advantage is that we can run our architecture in real-time saving computations while keeping high-quality results. ISSN:1522-4880

We introduce a novel algorithm that turns a flash selfie taken with a smartphone into a studio-like photograph with uniform lighting. Our method uses a convolutional neural network trained on a set of pairs of photographs acquired in a controlled environment. For each pair, we have one photograph of a subject's face taken with the camera flash enabled and another one of the same subject in the same pose illuminated using a photographic studio-lighting setup. We show how our method can amend lighting artifacts introduced by a close-up camera flash, such as specular highlights, shadows, and skin shine.

We present a method for turning a flash selfie taken with a smartphone into a photograph as if it was taken in a studio setting with uniform lighting. Our method uses a convolutional neural network trained on a set of pairs of photographs acquired in an ad-hoc acquisition campaign. Each pair consists of one photograph of a subject’s face taken with the camera flash enabled and another one of the same subject in the same pose illuminated using a photographic studio-lighting setup. We show how our method can amend defects introduced by a close-up camera flash, such as specular highlights, shadows, skin shine, and flattened images.

In the last years, the development of compression algorithms for image collections (e.g., photo albums) has become very popular due to the enormous diffusion of digital photographs. Typically, current solutions create an image sequence from images of the photo album to make them suitable for compression using a High Performance Video Coding (HEVC) encoder. In this study, we investigated a different approach to compress a collection of similar images. Our main idea is to exploit the inter- and intra- patch redundancy to compress the entire set of images. In practice, our approach is equivalent to compress the image set with Vector Quantization (VQ) using a global codebook. Our tests show that our clusterization algorithm is effective for a large number of images.

In this paper, we present a new technique for displaying High Dynamic Range (HDR) images on Low Dynamic Range (LDR) displays in an efficient way on the GPU. The described process has three stages. First, the input image is segmented into luminance zones. Second, the tone mapping operator (TMO) that performs better in each zone is automatically selected. Finally, the resulting tone mapping (TM) outputs for each zone are merged, generating the final LDR output image. To establish the TMO that performs better in each luminance zone we conducted a preliminary psychophysical experiment using a set of HDR images and six different TMOs. We validated our composite technique on several (new) HDR images and conducted a further psychophysical experiment, using an HDR display as the reference that establishes the advantages of our hybrid three-stage approach over a traditional individual TMO. Finally, we present a GPU version, which is perceptually equal to the standard version but with much improved computational performance.

International audience; High dynamic range (HDR) imaging enables to capture the full range of physical luminance of a real-world scene, and is expected to progressively replace traditional low dynamic range (LDR) pictures and videos. Despite the increasing HDR popularity, very little attention has been devoted to new forensic problems that are characteristic to this content. In this paper, we address for the first time such kind of problem, by identifying the source of an HDR picture. Specifically, we consider the two currently most common techniques to generate an HDR image: by fusing multiple LDR images with different exposure time, or by inverse tone mapping an LDR picture. We show that, in order to apply conventional forensic tools to HDR images, they need to be properly preprocessed, and we propose and evaluate a few simple HDR forensic preprocessing strategies for this purpose. In addition, we propose a new forensic feature based on Fisher scores, calculated under Gaussian mixture models. We show that the proposed feature outperforms the popular SPAM features in classifying the HDR image source on image blocks as small as 3 × 3, which makes our method suitable to detect composite forgeries combining HDR patches originating from different acquisition processes.

Few tone mapping operators (TMOs) take color management into consideration, limiting compression to luminance values only. This may lead to changes in image chroma and hues which are typically managed with a post-processing step. However, current post-processing techniques for tone reproduction do not explicitly consider the target display gamut. Gamut mapping on the other hand, deals with mapping images from one color gamut to another, usually smaller, gamut but has traditionally focused on smaller scale, chromatic changes. In this context, we present a novel gamut and tone management framework for color-accurate reproduction of high dynamic range (HDR) images, which is conceptually and computationally simple, parameter-free, and compatible with existing TMOs. In the CIE LCh color space, we compress chroma to fit the gamut of the output color space. This prevents hue and luminance shifts while taking gamut boundaries into consideration. We also propose a compatible lightness compression scheme that minimizes the number of color space conversions. Our results show that our gamut management method effectively compresses the chroma of tone mapped images, respecting the target gamut and without reducing image quality.

Capturing HDR videos is still a challenging task because HDR camera systems are either very costly (commercial systems) or have not matured enough to reach consumer markets (research prototypes). Recently, unofficial firmware for cameras has started to appear, and the firmware allows for reprogramming of the system to capture footage with an extended dynamic range. The dynamic range that can be captured in a single shot, however, is still limited and does not fully cover the dynamic range found in high-contrast scenes. There is, therefore, a need for low-cost techniques that allow for HDR-video capture using off-the-shelf equipment. This chapter describes methods for enhancing the standard dynamic range using previously captured HDR photographs of the same scene.

International audience; High dynamic range (HDR) imaging is attracting an increasing dealof attention in the multimedia community, yet its forensic problemshave been little studied so far. This paper proposes an HDR imageforensic method, which aims at differentiating HDR images createdfrom multiple low dynamic range (LDR) images from those createdfrom a single LDR image by inverse tone mapping. For each kindof HDR image, a Gaussian mixture model is learned. Thereafter,an HDR image forensic feature is constructed based on calculatingthe Fisher scores. With comparison to a steganalytic feature and atexture/facial analysis feature, experimental results demonstrate theefficiency of the proposed method in HDR image forensic classificationon whole images as well as small blocks, for three inverse tonemapping methods.

We introduce a novel algorithm for obtaining High Dynamic Range (HDR) videos from Standard Dynamic Range (SDR) videos with varying shutter speed or ISO per frame. This capturing technique represents today one of the most pop- ular HDR video acquisition methods, thanks to the avail- ability and the low cost of the equipment required; i.e., an off-the-shelf DSLR camera. However, na Ì^ıvely merging SDR frames into an HDR video can produce artifacts such as ghosts (when the scene is dynamic), and blurry edges (when the camera moves). In this work, we present a straightfor- ward, easy to implement, and fast technique that produces reasonable results in a short time. This is key for having quick previews of the captured videos without waiting for a long processing time. This is extremely important, espe- cially when capturing videos on modern mobile devices such as smartphones and/or tablets.

The ability to interactively render dynamic scenes with global illumination is one of the main challenges in computer graphics. The improvement in performance of interactive ray tracing brought about by significant advances in hardware and careful exploitation of coherence has rendered the potential of interactive global illumination a reality. However, the simulation of complex light transport phenomena, such as diffuse interreflections, is still quite costly to compute in real time. In this paper we present a caching scheme, termed Instant Caching, based on a combination of irradiance caching and instant radiosity. By reutilising calculations from neighbouring computations this results in a speedup over previous instant radiosity-based approaches. Additionally, temporal coherence is exploited by identifying which computations have been invalidated due to geometric transformations and updating only those paths. The exploitation of spatial and temporal coherence allows us to achieve superior frame rates for interactive global illumination within dynamic scenes, without any precomputation or quality loss when compared to previous methods; handling of lighting and material changes are also demonstrated., We thank Greg Ward for the Office and Conference scenes from Radiance package, Marko Dabrovic for the Sibenik cathedral model. Finally, we thank Stanford's Graphics Group for the Bunny model from the Stanford 3D Repository. We would like to thank Kadi Bouatouch for insightful comments on the document. This research was partially funded PT-FCT grant PTDC/EIA/65965/2006 (IGIDE project) and UK-EPSRC grant EP/D069874/2.

In the last few years, researchers in the field of High Dynamic Range (HDR) Imaging have focused on providing tools for expanding Low Dynamic Range (LDR) content for the generation of HDR images due to the growing popularity of HDR in applications, such as photography and rendering via Image-Based Lighting, and the imminent arrival of HDR displays to the consumer market. LDR content expansion is required due to the lack of fast and reliable consumer level HDR capture for still images and videos. Furthermore, LDR content expansion, will allow the re-use of legacy LDR stills, videos and LDR applications created, over the last century and more, to be widely available. The use of certain LDR expansion methods, those that are based on the inversion of Tone Mapping Operators (TMOs), has made it possible to create novel compression algorithms that tackle the problem of the size of HDR content storage, which remains one of the major obstacles to be overcome for the adoption of HDR. These methods are used in conjunction with traditional LDR compression methods and can evolve accordingly. The goal of this report is to provide a comprehensive overview on HDR Imaging, and an in depth review on these emerging topics.

In recent years inverse tone mapping techniques have been proposed for enhancing low-dynamic range (LDR) content for a high-dynamic range (HDR) experience on HDR displays, and for image based lighting. In this paper, we present a psychophysical study to evaluate the performance of inverse (reverse) tone mapping algorithms. Some of these techniques are computationally expensive because they need to resolve quantization problems that can occur when expanding an LDR image. Even if they can be implemented efficiently on hardware, the computational cost can still be high. An alternative is to utilize less complex operators; although these may suffer in terms of accuracy. Our study investigates, firstly, if a high level of complexity is needed for inverse tone mapping and, secondly, if a correlation exists between image content and quality. Two main applications have been considered: visualization on an HDR monitor and image-based lighting.

When colourimetrically characterising a high dynamic range display (HDR) built from an LCD panel and an LED backlight one is faced with several problems: the channels may not be constant; they may not be independent and there may be a significant radiant output at the black level. But crucially, colour transforms are underdetermined, which means that the number of colourimetric dimensions is smaller than the number of device channels. While the first three problems are associated with the LCD, the fourth problem stems from the additional channel in the HDR, the backlight. A 37'' flat-panel Brightside DR37-P HDR display was characterised. Using a spectroradiometer we recorded spectral radiance, chromaticities and luminance and estimated the true increase in gamut of the display due to the additional LED layer. We present a basic characterisation, propose a method for accurately presenting a desired luminance and chromaticity output despite the underdetermined problem and give an estimate of the available gamut.

In recent years many tone mapping operators (TMOs) have been presented in order to display high dynamic range images (HDRI) on typical display devices. TMOs compress the luminance range while trying to maintain contrast. The inverse of tone mapping, inverse tone mapping, expands a low dynamic range image (LDRI) into an HDRI. HDRIs contain a broader range of physical values that can be perceived by the human visual system. We propose a new framework that approximates a solution to this problem. Our framework uses importance sampling of light sources to find the areas considered to be of high luminance and subsequently applies density estimation to generate an expand map in order to extend the range in the high luminance areas using an inverse tone mapping operator. The majority of today’s media is stored in the low dynamic range. Inverse tone mapping operators (iTMOs) could thus potentially revive all of this content for use in high dynamic range display and image based lighting (IBL). Moreover, we show another application that benefits quick capture of HDRIs for use in IBL.

Photo-realistic rendering of virtual objects into real scenes is one of the most important research problems in computer graphics. Methods for capture and rendering of mixed reality scenes are driven by a large number of applications, ranging from augmented reality to visual effects and product visualization. Recent developments in computer graphics, computer vision, and imaging technology have enabled a wide range of new mixed reality techniques including methods of advanced image based lighting, capturing spatially varying lighting conditions, and algorithms for seamlessly rendering virtual objects directly into photographs without explicit measurements of the scene lighting. This report gives an overview of the state-of-the-art in this field, and presents a categorization and comparison of current methods. Our in-depth survey provides a tool for understanding the advantages and disadvantages of each method, and gives an overview of which technique is best suited to a specific problem. VPS

High dynamic range (HDR) displays use local backlight modulation to produce both high brightness levels and large contrast ratios. Thus, the display rendering algorithm and its parameters may greatly affect HDR visual experience. In this paper, we analyze the impact of display rendering on perceived quality for a specific display (SIM2 HDR47) and for a popular application scenario, i.e., HDR image compression. To this end, we assess whether significant differences exist between subjective quality of compressed images, when these are displayed using either the built-in rendering of the display, or a rendering algorithm developed by ourselves. As a second contribution of this paper, we investigate whether the possibility to estimate the true pixel-wise luminance emitted by the display, offered by our rendering approach, can improve the performance of HDR objective quality metrics that require true pixel-wise luminance as input. © (2015) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.

A new method for interactive rendering of complex lighting effects combines two algorithms. The first performs accurate ray tracing in heterogeneous refractive media to compute high-frequency phenomena. The second applies lattice-Boltzmann lighting to account for low-frequency multiple-scattering effects. The two algorithms execute in parallel on modern graphics hardware. This article includes a video animation of the authors' real-time algorithm rendering a variety of scenes.

International audience; —The goal of this paper is to assess whether existing solutions to expand low dynamic range (LDR) still images to high dynamic range (HDR) displays can apply to moving pictures. To this end, we conduct a subjective quality assessment test on four state-of-the-art expansion operators, using high-quality input LDR content. Our results show that, perhaps surprisingly, no temporal artifacts, such as flickering due to global illumination changes, are introduced applying frame-based algorithms. In addition, they confirm previous studies in that operators as simple as linear expansion can yield acceptable (or even excellent) quality of the HDR video.

Automatic exposure (AE) is a common method implemented in consumer cameras to calculate the best exposure that tries to minimize under and over exposed pixels for a scene. We present a generalized framework for an AE algorithm for high dynamic range (HDR) imaging inspired by tone mapping operators (TMO) and pointing devices. Compared to other methods such as [Neumann et al. 1998] we propose a more general framework and an approach that enables exploring images and videos in different areas of the image/frame.

The present paper proposes a novel watermarking scheme specifically designed for high dynamic range (HDR) images. The employed embedding strategy is based on a decomposition of the original HDR representation into multiple low dynamic range (LDR) images by means of a bracketing process. After having inserted the selected watermark into each LDR component, the final output is generated by combining the available contributions into a single HDR object. By exploiting some of the well studied properties of digital watermarking for standard LDR images, our approach is able to generate a watermarked HDR image visually equivalent to the original one, while allowing to detect the embedded information in both the marked HDR image and in its LDR counterpart, obtained through tone-mapping operators or by extracting a specific luminance range of interest from it. Several results obtained from an extensive set of experimental tests are reported to testify the effectiveness of the proposed scheme.

In this paper, we present EnvyDepth, an interface for recovering local illumination from a single HDR environment map. In EnvyDepth, the user quickly indicates strokes to mark regions of the environment map that should be grouped together in a single geometric primitive. From these annotated strokes, EnvyDepth uses edit propagation to create a detailed collection of virtual point lights that reproduce both the local and the distant lighting effects in the original scene. When compared to the sole use of the distant illumination, the added spatial information better reproduces a variety of local effects such as shadows, highlights and caustics. Without the effort needed to create precise scene reconstructions, EnvyDepth annotations take only tens of seconds to produce a plausible lighting without visible artifacts. This is easy to obtain even in the case of complex scenes, both indoors and outdoors. The generated lighting environments work well in a production pipeline since they are efficient to use and able to produce accurate renderings.

In this paper, we present BoostHDR, a novel method for compressing high dynamic range (HDR) images. Thealgorithm leverages on a novel segmentation-based tone mapping operator (TMO) which relaxes the no seamsconstraint. Our method can work with both JPEG and JPEG2000 encoders. Moreover, it provides better resultscompared to the state of the art in HDR images compression algorithms in terms of bit per pixels (bpp), andvisual quality using objective metrics.Keywords: HDR Imaging, HDR Image Compression 1. INTRODUCTION The HDR content capturing is now becoming very popular, allowing consumers to capture HDR images withcompact and DLSR cameras or even mobile phones. Moreover, DSLRs can be used for capturing HDR videos, 12 and HDR video cameras are starting to emerge. 6,20,27 This new era in capturing allows users to represent thefull luminance range that the human visual system (HVS) can perceive.On the other hand, HDR content requires more memory for storing the extra dynamic range information thanconventional imaging at 8-bit or low dynamic range (LDR) imaging. For example, an uncompressed HDR pixel,represented using 32-bit oating point, can require four times the amount of memory of an uncompressed LDRpixel at 8-bit. This can negatively a ect performances too, because more bandwidth is needed. For instance, itwould be prohibitive to manage a photographic gallery of uncompressed HDR images or to play HDR videos.In recent years, algorithms for HDR content memory compression have been proposed. These methods aretypically based on existing compression standards such as JPEG, JPEG2000, and MPEG, which can be modi edor extended to handle HDR information. However, the community has not agreed on a common standardencoding yet. This is quite critical, because the HDR imaging has reached the market as extra feature or appfor cameras and mobile phones.In this paper, we present a novel compression algorithm which is based on an segmentation-based TMO, andexisting image compression standards such as JPEG and JPEG2000. Our key contributions are: Backward compatibility : the information in compressed images can be visualized by a JPEG or JPEG2000standard viewer. This allows users to visualize part of the content when software that can decode ourcompression algorithm is not present. Improvement over state of the art : our proposed solution provides better performances in terms of visualquality and bpp than JPEG-HDR compression by Ward and Simmmons

In this paper we present a new technique for the display of High Dynamic Range (HDR) images on Low Dynamic Range (LDR) displays. The described process has three stages. First, the input image is segmented into luminance zones. Second, the tone mapping operator (TMO) that performs better in each zone is automatically selected. Finally, the resulting tone mapping (TM) outputs for each zone are merged, generating the final LDR output image. To establish the TMO that performs better in each luminance zone we conducted a preliminary psychophysical experiment using a set of HDR images and six different TMOs. We validated our composite technique on several (new) HDR images and conducted a further psychophysical experiment, using an HDR display as reference, that establishes the advantages of our hybrid three-stage approach over a traditional individual TMO.

In this work we present a new algorithm for accelerating the colour bilateral filter based on a subsampling strategy working in the spatial domain. The base idea is to use a suitable subset of samples of the entire kernel in order to obtain a good estimation of the exact filter values. The main advantages of the proposed approach are that it has an excellent trade-off between visual quality and speed-up, a very low memory overhead is required and it is straightforward to implement on the GPU allowing real-time filtering. We show different applications of the proposed filter, in particular efficient cross-bilateral filtering, real-time edge-aware image editing and fast video denoising. We compare our method against the state of the art in terms of image quality, time performance and memory usage. © 2012 Wiley Periodicals, Inc.

Retargeting refers to the process by which an image or video is adapted from the display device for which it was meant (target display) to another one (retarget display). The retarget display has different features from the target one such as dynamic range, discretization levels, color gamut, multi-view, and refresh rate spatial resolution. This is a very relevant topic in graphics, given the increasing number of display devices from large, high-contrast screens to small cell phones with limited dynamic range; a lot of techniques are being published in different venues, and it's hard to keep up. For most cases retargeting can be an ill-posed problem, for example in the process of displaying Low Dynamic Range (LDR) or 8-bit content on High Dynamic Range (HDR) displays. Such a problem requires the retargeting algorithm to generate new content which is missing in the input image/frame. In this course, we will present the latest solutions and techniques for retargeting images along various dimensions such as dynamic range, colors, temporal and spatial resolutions, and for the first time offer a much-needed holistic view of the field. Moreover, we are going to show how to measure and analyze the changes applied to an image or video in terms of quality using both psychophysical experiments (subjective) and computational metrics (objective). The course should be of interest to anyone involved in graphics in a broader sense, given the almost unavoidable need to retarget results to different devices -from developers interested in implementing retargeting techniques, to users that just need an overall perspective. For researchers fully engaged in developing multi-dimensional retargeting techniques, this course will serve as a solid background for future algorithms.

This poster presents a practical system for enhancing the quality of Low Dynamic Range (LDR) videos using High Dynamic Range (HDR) background images. Our technique relies on the assumption that the HDR information is static in the video footage. This assumption can be valid in many scenarios where moving subjects are the main focus of the footage and do not have to interact with moving light sources or highly reflective objects. Another valid scenario is teleconferencing via webcams, where the background is typically over-exposed, not allowing the users to perceive correctly the environment where the communication is happening.

The natural world presents our visual system with a wide, ever-changing range of colors and intensities. Existing video cameras are only capable of capturing a limited part of this wide range with sufficient resolution. High-dynamic-range (HDR) images can represent most of the real world's luminances, but until now capturing HDR images with a linear-response function has been limited to static scenes. This demonstration showcases a novel complete HDR video solution. The system includes a unique HDR video camera capable of capturing a full HDTV video stream consisting of 20 f-stops dynamic range at a resolution of 1920 x 1080 pixels at 30 frames per second; an encoding method for coping with the huge amount of data generated by the camera (achieving a compression ratio of up to 100:1 and real-time decompression); and a new 22-inch desktop HDR display for directly visualizing the dynamic HDR content.

In this paper we introduce an algorithm and related methods that expand the contrast range of Low Dynamic Range (LDR) videos in order to regenerate missing High Dynamic Range (HDR) data. For content generated from single exposure LDR sequences, this is clearly an under constrained problem. We achieved the expansion by inverting established tone mapping operator, a process we term inverse tone mapping. This approach is augmented by a number of methods which help expand the luminance for the required pixels while avoiding artifacts. These methods may be used to convert the large libraries of available legacy LDR content for use, for instance, on new content-starved HDR devices. Moreover, these same methods may be used to provide animated emissive surfaces for image based lighting (IBL). We demonstrate results for all the above applications and validate the resultant HDR videos with original HDR references using the HDR Visual Difference Predictor (HDR-VDP) image metric.

In recent years many Tone Mapping Operators (TMOs) have been presented in order to display High Dynamic Range Images (HDRI) on typical display devices. TMOs compress the luminance range while trying to maintain contrast. The dual of tone mapping, inverse tone mapping, expands a Low Dynamic Range Image (LDRI) into a HDRI. HDRIs contain a broader range of physical values that can be perceived by the human visual system. The majority of today's media is stored in low dynamic range. Inverse Tone Mapping Operators (iTMOs) could thus potentially revive all of this content for use in high dynamic range display and image-based lighting. We propose an approximate solution to this problem that uses median-cut to find the areas considered of high luminance and subsequently apply a density estimation to generate an Expand-map in order to extend the range in the high luminance areas using an inverse Photographic Tone Reproduction operator.

Realistic rendering of natural materials such as marble, jade, human skin, etc. requires the simulation of subsurface scattering of light. Such high-fidelity rendering, however, takes a significant amount of computation time, precluding its use in real-time application. This paper introduces a new technique for rendering the appearance of translucent material in real-time based on spherical harmonics. The technique consists of two passes, first the irradiance of the object is projected on a spherical harmonic basis, then in the second pass the exitant radiance is evaluated. This technique is not physically based, but it ensures high frame rates. It also supports deformable objects and with no precomputation time, so it could be used in interactive applications including videogames.

Translucency is an important feature for the representation of somereal world materials such as marble, jade, human skin and etc. Wepresent a new technique for rendering the appearance of translucentmaterial in real-time based on an image based approach. This tech-nique isn’t physically based as the dipole approximation [Jensenet al. 2001], but it reaches high frame rates, so it could be used ininteractive applications.

High-fidelity image synthesis is the process of computing images that are perceptually indistinguishable from the real world they are attempting to portray. Such a level of fidelity requires that the physical processes of materials and the behavior of light are accurately simulated. Most computer graphics algorithms assume that light passes freely between surfaces within an environment. However, in many applications, we also need to take into account how the light interacts with media, such as dust, smoke, fog, etc., between the surfaces. The computational requirements for calculating the interaction of light with such participating media are substantial. This process can take many hours and rendering effort is often spent on computing parts of the scene that may not be perceived by the viewer. In this paper, we present a novel perceptual strategy for physically based rendering of participating media. By using a combination of a saliency map with our new extinction map (X map), we can significantly reduce rendering times for inhomogeneous media. The visual quality of the resulting images is validated using two objective difference metrics and a subjective psychophysical experiment. Although the average pixel errors of these metric are all less than 1%, the subjective validation indicates that the degradation in quality still is noticeable for certain scenes. We thus introduce and validate a novel light map (L map) that accounts for salient features caused by multiple light scattering around light sources.