Search

Shadow detection is a common computer vision task that identifies the location of shadows. It has been used as a key primitive for many high-level tasks such as object segmentation and video tracking. Due to the complexity and variety of shadows, it is challenging to design a one-size-fits-all shadow detection algorithm in real-world scenarios. Instead, it is often necessary in practice to pick the best shadow detection algorithm for a particular vision application. While shadow detection algorithms can be assessed by comparing the detected shadow with the ground truth, such ground truth is often unavailable in real-world vision systems consistently generating new data. In this paper, we propose a no-reference shadow detection quality assessment network (NSDQA-Net) to rank shadow detection algorithms without needing the ground truth. For a given image, the proposed assessment network first learns a reference map that compensates the false positives and false negatives of multiple input shadow detection algorithms. The reference map, as well as the original image and the input shadow detection results, are then used to regress the quality scores of the shadow detection methods. Our framework supports the assessment of one, two, or multiple algorithms in a single run, making it easier for deployment in practical scenarios. Experiments results show that the proposed network achieves optimal accuracy in no-reference shadow detection assessment. Relying on it for adaptive algorithm selection also improves the performance of shadow detection using a fixed algorithm on benchmark datasets. [Display omitted] • We identify a task of no-reference quality assessment in shadow detection. • We design a novel no-reference shadow detection quality assessment network (NSDQA-Net). • We experimentally demonstrate the state-of-the-art performance of the proposed network. [ABSTRACT FROM AUTHOR]

We propose a novel intrinsic image decomposition method based on a single RGB-D image. We first separate the shading image into an illumination color component, a distant shading component and a local shading component, inducing a novel intrinsic image model that can encode color and spatial variation of scene illumination. Unlike previous methods, which assume illumination color is white, our light mixture model encodes scene illumination with two different light types, and an automatical strategy is proposed to calculate the color of the two light types. We also adopt physical-based illumination prior to infer the distant shading component. To do so, we firstly recover the illumination distribution of the distant light sources through solving a system of linear equations with sparse and non-negative constraints. Then, the recovered illumination is used to synthesize a coarse distant shading image jointly with the depth map. Later, the synthetic image is employed as an additional constraint of distant shading component. To reduce noise disturbance from the synthetic distant shading image, a novel sampling strategy was proposed. Finally, we consider the similarity of material locally and globally, which gives reliable constraints to the reflectance component. Experimental results demonstrate the validity and flexibility of our approach. [ABSTRACT FROM AUTHOR]

Abstract: We propose a novel method for relighting the image of outdoor scenes viewed from a fixed camera based on a sparse set of images of the same scene under different illumination. Unlike previous methods which require capturing images under pre-designed lighting or employing the 3D model of the target objects, our method adopts the technique of basis image which encapsulates material and geometry information of the scene into one image. We present a new method to calculate the basis images of an outdoor scene based on the sampling images captured in different time of a day, the relighting images corresponding to new sunlight incidence directions with arbitrary intensity can then be generated with these basis images. Besides, the subordinate shadow effect adhering to the sun's movement is also simulated, producing a visually plausible relighting result. Experiments demonstrate the efficiency and validity of our approach. [Copyright &y& Elsevier]

We propose a novel approach to simulate the illumination of augmented outdoor scene based on a legacy photograph. Unlike previous works which only take surface radiosity or lighting related prior information as the basis of illumination estimation, our method integrates both of these two items. By adopting spherical harmonics, we deduce a linear model with only six illumination parameters. The illumination of an outdoor scene is finally calculated by solving a linear least square problem with the color constraint of the sunlight and the skylight. A high quality environment map is then set up, leading to realistic rendering results. We also explore the problem of shadow casting between real and virtual objects without knowing the geometry of objects which cast shadows. An efficient method is proposed to project complex shadows (such as tree's shadows) on the ground of the real scene to the surface of the virtual object with texture mapping. Finally, we present an unified scheme for image composition of a real outdoor scene with virtual objects ensuring their illumination consistency and shadow consistency. Experiments demonstrate the effectiveness and flexibility of our method. [ABSTRACT FROM AUTHOR]

Abstract: In augmented reality, consistent illumination is an important aspect of integrating a virtual object into a video of a real scene. In this paper, we propose a novel image-based framework to estimate online the dynamically changing illumination parameters of outdoor video sequences captured by a fixed or pure rotation camera. Unlike previous approaches, which either require knowledge of the scene geometry or significant storage to preserve time- and view-dependent basis images or statistical parameters, our approach requires a very simple interaction at the initialization stage: a few brushes are used to select areas with a specified normal surface, and these areas are used to calculate the sunlight parameters. Additional areas with the same normal parameters will be detected automatically, and an optimization procedure is applied to ensure the robustness and precision of our estimation. Our new approach adapts to videos of the same scene with varied sun positions and camera panning. The experimental results demonstrate the effectiveness and flexibility of our proposed approach. [Copyright &y& Elsevier]

Hypertensive retinopathy (HR) can potentially lead to vision loss if left untreated. Early screening and treatment are critical in reducing the risk of vision loss. The computer-aided diagnostic system presents an opportunity to improve the efficiency and reliability of HR screening and diagnosis, particularly given the shortage of specialized medical professionals and the challenges faced by primary care physicians in making precise diagnoses. A notable barrier to the development of such diagnostic algorithms is the lack of publicly available benchmarks and datasets. To address these issues, we organized a challenge named “HRDC—Hypertensive Retinopathy Diagnosis Challenge” in conjunction with the Computer Graphics International (CGI) 2023 conference. The challenge provided a fundus image dataset for two clinical tasks: hypertension classification and HR classification, with each task containing 1000 images. This paper presents a concise summary and analysis of the submitted methods and results for the two challenge tasks. For hypertension classification, the best performing algorithm achieved a Kappa score of 0.3819, an F1 score of 0.6337, and a specificity of 0.8472. For HR classification, the best performing algorithm achieved a Kappa score of 0.4154, an F1 score of 0.6122, and a specificity of 0.8444. We also explored an ensemble approach to the top-ranking methods, which further improved performance beyond the individual best performing algorithm for each task. The challenge results show that there is room for further optimization of these methods, but the insights and methodologies derived from this challenge provide valuable directions for developing more precise and reliable classification models for hypertension and HR. [ABSTRACT FROM AUTHOR]

• An illumination-guided plane fitting algorithm for Virtual Planar Area Lights generation. • A normal correction factor to reduce the illumination errors introduced by plane fitting. • A virtual light sources reuse strategy in order to address the flickering artifacts. [Display omitted] This paper presents a method called Virtual Planar Area Lights (VPALs) to generate single-bounce diffuse-to-diffuse indirect illumination for dynamic scenes and light sources. Different from the VPLs (Virtual Point Lights) based methods, which extract point lights from RSM (Reflective Shadow Map), our method fits RSM by many planar regions which will be directly used as virtual light sources to generate the indirect illumination. In order to reduce the illumination errors introduced by plane fitting, we propose several strategies, including an illumination-guided plane fitting algorithm, a normal correction factor and a virtual light sources reuse strategy. Our experimental results indicate that our method can achieve a good balance between the rendering performance and quality. Compared with traditional light culling and clustering methods, the rendering consumption of our algorithm is reduced by at least 67 % with the similar RMSE (Root Mean Squared Error). [ABSTRACT FROM AUTHOR]

• A complete pipeline to approximate single-bounce diffuse-to-diffuse indirect illumination in the foveal region. • An efficient method to evaluate VPLs contribution to final image quality. • Generate an importance map to guide the sampling of shading points. • A fast visibility estimation method to accelerate VPLs contribution computation. [Display omitted] In this paper, we propose a novel Foveated Light Culling (FLC) method to efficiently approximate global illumination for foveated rendering in virtual reality applications. The key idea is to cull the virtual point lights (VPLs) in accordance with their contribution to the visual perception, which makes the loss of details caused is imperceptible to the human eyes. Based on this idea and the features of human visual system (HVS), we calculate the contribution of each VPL according to samples generated from a perceptual importance map. By estimating the local visibility between VPLs and samples, our method can greatly improve the performance during the process of evaluating contributions. Our method is able to produce high-quality single-bounce diffuse-to-diffuse indirect illumination in the foveal region at real-time frame rate while supporting dynamic scenes and light sources. Furthermore, the experiments show that our approach makes a good balance between foveal quality and render efficiency. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel hybrid occlusion culling method for large scale scenes. The basic idea is to use an iterative hierarchical Z-buffer occlusion culling algorithm to execute a coarse-grained culling via compute shader, followed by a fine-grained culling via rasterization. We also propose a forward warping method to generate a low resolution approximated depth map to accelerate the culling process. Our solution requires only one indirect multidraw command and runs on GPU totally without any CPU read-back operations. Our experimental results indicate that our algorithm outperforms the existing solutions both in performance and culling rate. • We propose a hybrid occlusion culling method with hierarchical structure. • We further improve the performance of our algorithm by proposing a forward warping. • Our algorithm outperforms the existing solutions in performance and culling rate. [ABSTRACT FROM AUTHOR]

Illumination normalization for face recognition is of great importance when the face image is taken under a harsh lighting condition and only one single image is available. In this paper, an illumination normalization method that can preserve face identity and make the processed image have photorealistic face texture is proposed. First, an illumination regression filter is proposed to remove a large number of illumination components of the input image. Then, the accelerated proximal gradient algorithm is used for low-rank decomposition of the filtered image, which further reduces the residual illumination and noise of the face image. However, after these two operations, the face image has a peeled-off appearance. To avoid the unrealistic appearance, a deep image prior synthesis process is developed to synthesize the photorealistic face texture. To accomplish this task, the filtered image, the low-rank decomposed image, the original image and any a uniformly illuminated frontal face image as reference are used as synthetic ingredients. These ingredients, as prior knowledge, synthesize a face image that has the texture features of the reference image and retains the identity of the input image. We try to bridge the appearance gap between the synthetic ingredients and the final synthetic image by a deep neural network. Instead of training the network on a large dataset, the final synthetic image is generated by iterating the parameters of the generator-structured network according to a certain proportion of the synthetic ingredients. Systematic evaluations conducted on public databases demonstrate that the proposed method is robust to illumination with better performance than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Real-time camera calibration has been intensively studied in augmented reality. However, for texture-less and texture-repeated scenes as well as poorly illuminated scenes, obtaining a stable calibration is still an open problem. In the paper, we propose a method of calibrating a live video by tracking orthogonal vanishing points. Since vanishing points cannot be obtained directly on the image, the tracking is achieved by tracking parallel lines. This is a changeling problem due to the fact that vanishing points are sensitive to image noise, camera movement, and illumination variation. We tackle the challenges by three optimization procedures and flexible process of degenerated cases. During three optimizations, several explicitly geometric constraints are incorporated, ensuring the calibration result robust to poor illumination and camera movement. A variety of challenging examples demonstrate that the proposed algorithm outperforms state-of-the-art methods for texture-less and texture-repeated scenes. [ABSTRACT FROM AUTHOR]

In this paper, we proposed a method to recover the reflectance and shading images of face images, such as portraits, captured under complex lighting conditions. Under such lighting conditions, traditional intrinsic image decomposition can hardly be applied on face images especially with cast shadows. Inspired by the fact that the face symmetry is an inherent property of reflectance image for human face, face symmetry is firstly exploited as a constraint into traditional intrinsic decomposition framework to obtain the initial reflectance image. To remove the artifacts of the initial reflectance image caused by cast shadows, we use the skin color model to refine the reflectance image. Experiments show that our method is effective for face images under complex lighting conditions. [ABSTRACT FROM AUTHOR]

In augmented reality, it is essential that the rendered virtual objects are embedded harmonically into the view of the background scenes and their appearance should reflect the changing lighting condition of the real scene to ensure illumination consistency. In this paper, we propose a novel method to solve for the sunlight and skylight basis images of static outdoor scenes from a time-lapse image sequence. It is proved that the resulted basis images encapsulate the geometry and material reflectivity of the scene, correspond to the global illumination effects of the outdoor scene under a unit intensity of the sunlight and skylight. Our method is fully automatic. Unlike previous methods, it gets rid of the constraints that the reflectance of all objects in scenes should be ideal diffuse, or the weather condition should be overcast or sunshine. During decomposition, we first detect shadowed pixels by analyzing the time-lapse curve of each pixel through k-means clustering, the basis images of sunlight and skylight are then solved by an iterative procedure with the decomposition equation. The basis images are further optimized by exploiting their constraints and priors. Experimental results demonstrate the effectiveness and flexibility of the proposed method. Our method can also be applied in image understanding and compressing. [ABSTRACT FROM AUTHOR]

A correction to the article "Color face image decomposition under complex lighting conditions" that was published in a 2014 issue is presented.