Search

Copyright of Automotive Digest is the property of Automotive Digest Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The images we commonly use are RGB images that contain three pieces of information: red, green, and blue. On the other hand, hyperspectral (HS) images retain wavelength information. HS images are utilized in various fields due to their rich information content, but acquiring them requires specialized and expensive equipment that is not easily accessible to everyone. Recently, Spectral Super-Resolution (SSR), which generates spectral images from RGB images, has been studied. Conventional SSR methods target Low Dynamic Range (LDR) images. However, some practical applications require High Dynamic Range (HDR) images. In this paper, an SSR method for HDR is proposed. As a practical example, we use the HDR-HS images generated by the proposed method as environment maps and perform spectral image-based lighting. The rendering results by our method are more realistic than conventional renderers and LDR SSR methods, and this is the first attempt to utilize SSR for spectral rendering. [ABSTRACT FROM AUTHOR]

The complex behavior of metallic and pearlescent paints is not accurately captured with simple (spatially varying) BRDF models. They exhibit goniochromatic effects and glints due to metallic flakes embedded in the paint. However, when viewed from a distance or when illuminated from a larger area light source, the high-frequency glints tend to vanish. Therefore, we do not consider glints in this work and focus on the remaining angular color variations of a homogeneous materials, based on the direction from which the paint is illuminated or viewed, and how to realize these effects in a real-time rendering engine. Many models have been proposed to capture these effects, but efficiently and accurately reproducing them in real-time rendering applications under image-based lighting is a challenging problem. Therefore, most of the analytical models are not suitable either because of their computational complexity or because they are not able to present these effects accurately enough. We propose a prefiltering technique to handle angular color variations with very little overhead in comparison to the widely used standard methods for real-time image-based lighting. We require that the BRDF can be factorized into a brightness BRDF which works well with real-time image-based lighting, and a function modeling the angular color variations. The color is then prefiltered with a kernel defined by the environment map and brightness BRDF. Marginalizing over the azimuthal rotation around the ideal reflection direction results in a 3D prefiltered color table, parameterized by the ideal reflection direction and the angle between viewing direction and surface normal, which can further be decomposed into two functions depending on one of the two parameters, respectively. The additional data that needs to be stored with our method consists of a low-resolution cubemap and 2D array per material and environment map combination. [Display omitted] • Render real-world goniochromatic materials in real time. • Image-based lighting formulated as filtering color term with illumination and brightness BRDF. • Low storage requirements and low computational overhead at runtime. [ABSTRACT FROM AUTHOR]

The images we commonly use are RGB images that contain three pieces of information: red, green, and blue. On the other hand, hyperspectral (HS) images retain wavelength information. HS images are utilized in various fields due to their rich information content, but acquiring them requires specialized and expensive equipment that is not easily accessible to everyone. Recently, Spectral Super-Resolution (SSR), which generates spectral images from RGB images, has been studied. Conventional SSR methods target Low Dynamic Range (LDR) images. However, some practical applications require High Dynamic Range (HDR) images. In this paper, an SSR method for HDR is proposed. As a practical example, we use the HDR-HS images generated by the proposed method as environment maps and perform spectral image-based lighting. The rendering results by our method are more realistic than conventional renderers and LDR SSR methods, and this is the first attempt to utilize SSR for spectral rendering.

Corresponding lighting and reflectance between real and virtual objects is important for spatial presence in augmented and mixed reality (AR and MR) applications. We present a method to reconstruct real-world environmental lighting, encoded as a reflection map (RM), from a conventional photograph. To achieve this, we propose a stacked convolutional neural network (SCNN) that predicts high dynamic range (HDR) 360 ${}^\circ$ ∘ RMs with varying roughness from a limited field of view, low dynamic range photograph. The SCNN is progressively trained from high to low roughness to predict RMs at varying roughness levels, where each roughness level corresponds to a virtual object’s roughness (from diffuse to glossy) for rendering. The predicted RM provides high-fidelity rendering of virtual objects to match with the background photograph. We illustrate the use of our method with indoor and outdoor scenes trained on separate indoor/outdoor SCNNs showing plausible rendering and composition of virtual objects in AR/MR. We show that our method has improved quality over previous methods with a comparative user study and error metrics. [ABSTRACT FROM AUTHOR]

A realistically inserted virtual object in the real-time physical environment is a desirable feature in augmented reality (AR) applications and mixed reality (MR) in general. This problem is considered a vital research area in computer graphics, a field that is experiencing ongoing discovery. The algorithms and methods used to obtain dynamic and real-time illumination measurement, estimating, and rendering of augmented reality scenes are utilized in many applications to achieve a realistic perception by humans. We cannot deny the powerful impact of the continuous development of computer vision and machine learning techniques accompanied by the original computer graphics and image processing methods to provide a significant range of novel AR/MR techniques. These techniques include methods for light source acquisition through image-based lighting or sampling, registering and estimating the lighting conditions, and composition of global illumination. In this review, we discussed the pipeline stages with the details elaborated about the methods and techniques that contributed to the development of providing a photo-realistic rendering, visual coherence, and interactive real-time illumination results in AR/MR. [ABSTRACT FROM AUTHOR]

Highlights • The accuracy of a horizontal hybrid photo-radiometer sky model is examined by comparing to the Perez all-weather sky model. • The application of the horizontal hybrid photo-radiometer sky model is extended and enriched within four office spaces. • The results indicate that both sky models generate luminance maps at the same level of accuracy. Abstract Daylight simulation results greatly rely on the accuracy of sky models. When generic CIE sky and Perez all-weather sky simulate daylighting scenes they simplify cloud distributions. The lack of cloud distributions and subtle luminance variations causes simulation result deviations. High Dynamic Range (HDR) image techniques provide a method of generating accurate and actual skies. This paper explores the accuracy of a horizontal hybrid phadiometer (HPR) sky model, which combines modeled physical descriptions of the sun and HDR sky images. The aim was to build on and enrich the findings of a previous study (Humann and McNeil, 2017). In this study, the HDR images within four office spaces were taken. Simultaneously, global horizontal illuminance and HDR sky images were recorded outside. Simulated luminance maps were generated under both the HPR and Perez skies. Daylight Glare Probability and vertical eye illuminance (E v) were calculated from 112 groups of luminance maps. The simulated E v under HPR and Perez skies presented relative RMSE of 21.3% and 23.7%, respectively. The frequencies of accurate glare prediction under HPR and Perez skies were 95.5% and 93.9%, respectively. The results indicated that both the horizontal HPR sky and the Perez sky simulate equally accurate luminance maps. [ABSTRACT FROM AUTHOR]

We present a practical framework for reproducing omnidirectional incident illumination conditions with complex spectra using a light stage with multispectral LED lights. For lighting acquisition, we augment standard RGB panoramic photography with one or more observations of a color chart with numerous reflectance spectra. We then solve for how to drive the multispectral light sources so that they best reproduce the appearance of the color charts in the original lighting. Even when solving for non-negative intensities, we show that accurate lighting reproduction is achievable using just four or six distinct LED spectra for a wide range of incident illumination spectra. A significant benefit of our approach is that it does not require the use of specialized equipment (other than the light stage) such as monochromators, spectroradiometers, or explicit knowledge of the LED power spectra, camera spectral response functions, or color chart reflectance spectra. We describe two simple devices for multispectral lighting capture, one for slow measurements of detailed angular spectral detail, and one for fast measurements with coarse angular detail. We validate the approach by realistically compositing real subjects into acquired lighting environments, showing accurate matches to how the subject would actually look within the environments, even for those including complex multispectral illumination. We also demonstrate dynamic lighting capture and playback using the technique. [ABSTRACT FROM AUTHOR]

High dynamic range (HDR) imaging techniques are nowadays widely used in building research to capture luminances in the occupant field of view and investigate visual discomfort. This photographic technique also makes it possible to map sky luminances. Such images can be used for illuminating virtual scenes; the technique is called image-based lighting (IBL). This paper presents a work investigating IBL in a lighting quality research context for accelerating the development of appearance-driven performance indicators. Simulations were carried out using Radiance software. The ability of IBL to accurately predict indoor luminances is discussed by comparison with luminances from HDR photographs and luminances predicted by simulation in modeling the sky in several other more traditional ways. The present study confirms previous observations that IBL leads to similar luminance values than far less laborious simulations in which the sky is modeled based on outdoor illuminance measurements. IBL and these last methods minimize differences with HDR photographs in comparison to sky modeling not based on outdoor measurements.

Photo-realistic rendering of virtual objects into real scenes is one of the most important research problems in computer graphics. Methods for capture and rendering of mixed reality scenes are driven by a large number of applications, ranging from augmented reality to visual effects and product visualization. Recent developments in computer graphics, computer vision, and imaging technology have enabled a wide range of new mixed reality techniques including methods for advanced image based lighting, capturing spatially varying lighting conditions, and algorithms for seamlessly rendering virtual objects directly into photographs without explicit measurements of the scene lighting. This report gives an overview of the state-of-the-art in this field, and presents a categorization and comparison of current methods. Our in-depth survey provides a tool for understanding the advantages and disadvantages of each method, and gives an overview of which technique is best suited to a specific problem. [ABSTRACT FROM AUTHOR]

This paper describes a framework for aerial imaging of high dynamic range (HDR) scenes for use in virtual reality applications, such as immersive panorama applications and photorealistic superimposition of virtual objects using image-based lighting. We propose a complete and practical system to acquire full spherical HDR images from the sky, using two omnidirectional cameras mounted above and below an unmanned aircraft. The HDR images are generated by combining multiple omnidirectional images captured with different exposures controlled automatically. Our system consists of methods for image completion, alignment, and color correction, as well as a novel approach for automatic exposure control, which selects optimal exposure so as to avoid banding artifacts. Experimental results indicated that our system generated better spherical images compared to an ordinary spherical image completion system in terms of naturalness and accuracy. In addition to proposing an imaging method, we have carried out an experiment about display methods for aerial HDR immersive panoramas utilizing spherical images acquired by the proposed system. The experiment demonstrated HDR imaging is beneficial to immersive panorama using an HMD, in addition to ordinary uses of HDR images. [ABSTRACT FROM AUTHOR]

We propose a novel painting interface that enables users to design an illumination distribution for a real room using an array of computer-controlled lights. Users specify which area of the room is to be well-lit and which is to be dark by painting a target illumination distribution on a tablet device displaying the image obtained by a camera mounted in the room. The painting result is overlaid on the camera image as contour lines of the target illumination intensity. The system then runs an optimization to calculate light parameters to deliver the requested illumination condition. We implemented a GPU-based parallel search to achieve real-time processing. In our system, we used actuated lights that can change the lighting direction to generate the requested illumination condition more faithfully than static lights. We built a miniature-scale experimental environment and ran a user study to compare our method with a standard direct manipulation method using sliders. The results showed that the users preferred our method for informal light control. [ABSTRACT FROM AUTHOR]

In this paper, we present EnvyDepth, an interface for recovering local illumination from a single HDR environment map. In EnvyDepth, the user quickly indicates strokes to mark regions of the environment map that should be grouped together in a single geometric primitive. From these annotated strokes, EnvyDepth uses edit propagation to create a detailed collection of virtual point lights that reproduce both the local and the distant lighting effects in the original scene. When compared to the sole use of the distant illumination, the added spatial information better reproduces a variety of local effects such as shadows, highlights and caustics. Without the effort needed to create precise scene reconstructions, EnvyDepth annotations take only tens of seconds to produce a plausible lighting without visible artifacts. This is easy to obtain even in the case of complex scenes, both indoors and outdoors. The generated lighting environments work well in a production pipeline since they are efficient to use and able to produce accurate renderings. [ABSTRACT FROM AUTHOR]

High-fidelity rendering can be used to investigate Cultural Heritage (CH) sites in a scientifically rigorous manner. However, a high degree of realism in the reconstruction of a CH site can be misleading insofar as it can be seen to imply a high degree of certainty about the displayed scene-which is frequently not the case, especially when investigating the past. So far, little effort has gone into adapting and formulating a Predictive Rendering pipeline for CH research applications. In this paper, we first discuss the goals and the workflow of CH reconstructions in general, as well as those of traditional Predictive Rendering. Based on this, we then propose a research framework for CH research, which we refer to as 'Cultural Heritage Predictive Rendering' (CHPR). This is an extension to Predictive Rendering that introduces a temporal component and addresses uncertainty that is important for the scene's historical interpretation. To demonstrate these concepts, two example case studies are detailed. [ABSTRACT FROM AUTHOR]

In mixed reality (MR) design review, the aesthetics of a virtual prototype is assessed by integrating a virtual model into a real-world environment and inspecting the interaction between the model and the environment (lighting, shadows and reflections) from different points of view. The visualization of the virtual model has to be as realistically as possible to provide a solid basis for this assessment and interactive rendering speed is mandatory to allow the designer to examine the scene from arbitrary positions. In this article we present a real-time rendering engine specifically tailored to the needs of MR visualization. The renderer utilizes pre-computed radiance transfer to calculate dynamic soft-shadows, high dynamic range images and image-based lighting to capture incident real-world lighting, approximate bidirectional texture functions to render materials with self-shadowing, and frame post-processing filters (bloom filter and an adaptive tone mapping operator). The proposed combination of rendering techniques provides a trade-off between rendering quality and required computing resources which enables high quality rendering in mobile MR scenarios. The resulting image fidelity is superior to radiosity-based techniques because glossy materials and dynamic environment lighting with soft-shadows are supported. Ray tracing-based techniques provide higher quality images than the proposed system, but they require a cluster of computers to achieve interactive frame rates which prevents these techniques from being used in mobile MR (especially outdoor) scenarios. The renderer was developed in the European research project IMPROVE (FP6-IST-2-004785) and is currently extended in the MAXIMUS project (FP7-ICT-1-217039) where hybrid rendering techniques which fuse PRT and ray tracing are developed. [ABSTRACT FROM AUTHOR]

Webcams placed all over the world observe and record the visual appearance of a variety of outdoor scenes over long periods of time. The recorded time-lapse image sequences cover a wide range of illumination and weather conditions -- a vast untapped resource for creating visual realism. In this work, we propose to use a large repository of webcams as a "clip art" library from which users may transfer scene appearance (objects, scene backdrops, outdoor illumination) into their own time-lapse sequences or ever single photographs. The goal is to combine the recent ideas from data-driven appearance transfer techniques with a general and theoretically-grounded physically-based illumination model. To accomplish this, the paper presents three main research contributions: 1) a new, high-quality outdoor webcam database that has been calibrated radiometrically and geometrically; 2) a novel approach for matching illuminations across different scenes based on the estimation of the properties of natural illuminants (sun, sky, weather and clouds), the camera geometry, and illumination-depended scene features; 3) a new algorithm for generating physically plausible high dynamic range environment maps for each frame in a webcam sequence. [ABSTRACT FROM AUTHOR]

Captured reflectance fields tend to provide a relatively coarse sampling of the incident light directions. As a result, sharp illumination features, such as highlights or shadow boundaries, are poorly reconstructed during relighting; highlights are disconnected, and shadows show banding artefacts. In this paper, we propose a novel interpolation technique for 4D reflectance fields that reconstructs plausible images even for non-observed light directions. Given a sparsely sampled reflectance field, we can effectively synthesize images as they would have been obtained from denser sampling. The processing pipeline consists of three steps: (1) segmentation of regions where intermediate lighting cannot be obtained by blending, (2) appropriate flow algorithms for highlights and shadows, plus (3) a final reconstruction technique that uses image-based priors to faithfully correct errors that might be introduced by the segmentation or flow step. The algorithm reliably reproduces scenes that contain specular highlights, interreflections, shadows or caustics. [ABSTRACT FROM AUTHOR]

We apply simplified image-based lighting methods to reduce the equipment, cost, time, and specialized skills required for high-quality photographic lighting of desktop-sized static objects such as museum artifacts, We place the object and a computer-steered moving-head spotlight inside a simple foam-core enclosure and use a camera to record photos as the light scans the box interior. Optimization, guided by interactive user sketching, selects a small set of these photos whose weighted sum best matches the user-defined target sketch. Unlike previous image-based relighting efforts, our method requires only a single area light source, yet it can achieve high-resolution light positioning to avoid multiple sharp shadows. A reduced version uses only a handheld light and may be suitable for battery-powered field photography equipment that fits into a backpack. [ABSTRACT FROM AUTHOR]

Photo editing software allows digital images to be blurred, warped or re-colored at the touch of a button. However, it is not currently possible to change the material appearance of an object except by painstakingly painting over the appropriate pixels. Here we present a method for automatically replacing one material with another, completely different material, starting with only a single high dynamic range image as input. Our approach exploits the fact that human vision is surprisingly tolerant of certain (sometimes enormous) physical inaccuracies, while being sensitive to others. By adjusting our simulations to be careful about those aspects to which the human visual system is sensitive, we are for the first time able to demonstrate significant material changes on the basis of a single photograph as input. [ABSTRACT FROM AUTHOR]

This paper presents a methodology for integrating synthetic objects into real scenes. We take a set of photographs of the real scene and build a simple image-based model. We use high dynamic range images to build an accurate representation of the lighting in the scene. Then we insert a synthetic object into the model and compute its illumination and shading using the lighting information. Illumination changes produced by the synthetic object are also applied to real-scene objects located nearby. We show how easy it is to achieve photo-realistic results without specialized hardware. Our approach takes advantage of techniques like automatic camera calibration, high dynamic range image capture and image-based lighting. [Copyright &y& Elsevier]