Search

Endoscopy images show part of the gastrointestinal tract or other parts of the human body. Due to the complex environment in which this tract is located in the human body and the limitations of the image acquisition equipment, endoscopy images may present blur and specular highlights that are common types of degradation. In this paper, we present a blind deblurring approach for endoscopy images that estimate the blur kernel from the fusion of the specular highlights. The proposed method can be divided into two phases. First, the specular highlights are precisely extracted from the original image using a sparse and low-rank image decomposition approach. Then, these highlights are automatically clustered and fused according to the spatial and intensity features. In this way, potential kernels are generated and used for the deblurring process. The experimental results show that the proposed method presents an improved and comparable performance when considered other state-of-the-art methods that use highlights segmentation and image deblurring.

The separation of reflection components is an important issue in computer graphics, computer vision, and image processing. This is a highly ill-posed problem since the number of unknowns to solve is much larger than the number of equations. We present a method to reduce the difficulty of this problem by assuming that surface geometry is known. A novel objective function based on robust principal component analysis is proposed to simultaneously separate specularities and estimate the position of light source. We develop an Augmented Lagrangian Multiplier based algorithm to solve the objective function efficiently and effectively. Experimental results on real-world and synthetic data demonstrate the effectiveness of our method.

Door detection is becoming an increasingly important subject in building indoor modelling owing to its value in scan-to-BIM processes. This paper presents an original approach that detects open, semi-open and closed doors in 3D laser scanned data of indoor environments. The proposed technique is unique in that it integrates the information regarding both the geometry (i.e. XYZ coordinates) and colour (i.e. RGB or HSV) provided by a calibrated set of 3D laser scanner and a colour camera. In other words, our technique is developed in a 6D-space framework. The geometry-colour integration and other characteristics of our method make it robust to occlusion and variations in colours resulting from varying lighting conditions at each scanning location (e.g. specular highlights) and from different scanning locations. In addition to this paper, the authors also contribute a public dataset of real scenes along with an annotated ground truth. The dataset has varying levels of challenges and will help to assess the performance of new and existing contributions in the field. The approach proposed in this paper is tested against that dataset, yielding encouraging results.

This paper presents a novel structured light approach for the 3D reconstruction of specular surface. The binary shifting strip is adopted as structured light pattern instead of conventional sinusoidal pattern. Based on the framework of conventional High Dynamic Range (HDR) imaging technique, an efficient means is first introduced to estimate the camera response function. And then, dynamic range of the generated radiance map is compressed in the gradient domain by introducing an attenuation function. Subject to the change of lighting conditions caused by projecting different structured light patterns, the structure light image with middle exposure level is selected as the reference image and used for the slight adjustment of the primary fused image. Finally, the regenerated structured light images with well exposing condition are used for 3D reconstruction of the specular surface. To evaluate performance of the method, some stainless stamping parts with strong reflectivity are used for the experiments. And the results showed that, different specular targets with various shapes can be precisely reconstructed by the proposed method.

A novel specular highlight restoration algorithm has been proposed to remove the specular highlight in a real-time vision system. In this paper, the specular highlight region has been detected and it has been restored by image inpainting method. Most of specular highlight detection algorithms are effective only for a single image. And the auto-threshold algorithm has been implemented in real time, but, it has still low reliability with a heavy computational cost. The proposed system detects pixels corresponding to the specular highlight region in the HSI color space through a newly defined classification table. So, Specular highlight can be detected quickly with these simplified classification table. In addition, it has versatility because it can be combined with various existing high-performance image inpainting method. The superiority of the proposed algorithm is compared with the conventional specular highlight removal algorithm by combining proposed algorithm with two high-performance image inpainting techniques.

In social interactions, our sense of when we have eye contact with another person relies on the distribution of luminance across their eye region, reflecting the position of the darker iris within the lighter sclera of the human eye. This distribution of luminance can be distorted by the lighting conditions, consistent with the fundamental challenge that the visual system faces in distinguishing the nature of a surface from the pattern of light falling upon it. Here we perform a set of psychophysics experiments in human observers to investigate how illumination impacts on the perception of eye contact. First, we find that simple changes in the direction of illumination can produce systematic biases in our sense of when we have eye contact with another person. Second, we find that the visual system uses information about the lighting conditions to partially discount or ‘explain away’ the effects of illumination in this context, leading to a significantly more robust sense of when we have eye contact with another person. Third, we find that perceived eye contact is affected by specular reflections from the eye surface in addition to shading patterns, implicating eye glint as a potential cue to gaze direction. Overall, this illustrates how our interpretation of social signals relies on visual mechanisms that both compensate for the effects of illumination on retinal input and potentially exploit novel cues that illumination can produce.

In this paper, we introduce and rigorously solve for the first time the photometric stereo problem for central panoramic systems. In particular, by leveraging the unified camera model, we present a new spherical image irradiance equation and analyze its mathematical properties in detail. The discrete gradient field issued from this equation is fed into a drift-free normal integration algorithm tailored to the spherical image geometry, and the light direction is estimated using the specular highlights observed on mirror balls inside the scene. Extensive experiments conducted with analytical surfaces, and synthetic and real-world images captured by central panoramic cameras, show that the proposed 3D reconstruction pipeline is effective and tolerant to noise. The image database and the code developed are publicly available on the Internet at the address: https://home.mis.u-picardie.fr/ fabio/ustereo.html

Computer simulated stimuli can provide a flexible method for creating artificial scenes in the study of visual perception of material surface properties. Previous work based on this approach reported that the properties of surface roughness and glossiness are mutually interdependent and therefore, perception of one affects the perception of the other. In this case roughness was limited to a surface property termed bumpiness. This paper reports a study into how perceived gloss varies with two model parameters related to surface roughness in computer simulations: the mesoscale roughness parameter in a surface geometry model and the microscale roughness parameter in a surface reflectance model. We used a real-world environment map to provide complex illumination and a physically-based path tracer for rendering the stimuli. Eight observers took part in a 2AFC experiment, and the results were tested against conjoint measurement models. We found that although both of the above roughness parameters significantly affect perceived gloss, the additive model does not adequately describe their mutually interactive and nonlinear influence, which is at variance with previous findings. We investigated five image properties used to quantify specular highlights, and found that perceived gloss is well predicted using a linear model. Our findings provide computational support to the 'statistical appearance models' proposed recently for material perception.

In dynamic scenes, relative motion between the object, the observer, and/or the environment projects as dynamic visual information onto the retina (optic flow) that facilitates 3D shape perception. When the object is diffusely reflective, e.g. a matte painted surface, this optic flow is directly linked to object shape, a property found at the foundations of most traditional shape-from-motion (SfM) schemes. When the object is specular, the corresponding specular flow is related to shape curvature, a regime change that challenges the visual system to determine concurrently both the shape and the distortions of the (sometimes unknown) environment reflected from its surface. While human observers are able to judge the global 3D shape of most specular objects, shape-from-specular-flow (SFSF) is not veridical. In fact, recent studies have also shown systematic biases in the perceived motion of such objects. Here we focus on the perception of local shape from specular flow and compare it to that of matte-textured rotating objects. Observers judged local surface shape by adjusting a rotation and scale invariant shape index probe. Compared to shape judgments of static objects we find that object motion decreases intra-observer variability in local shape estimation. Moreover, object motion introduces systematic changes in perceived shape between matte-textured and specular conditions. Taken together, this study provides a new insight toward the contribution of motion and surface material to local shape perception. (C) 2015 The Authors. Published by Elsevier Ltd.

We investigated whether specular highlights facilitate the perception of shape from shading in a search paradigm and how highlights interact with shading to facilitate this perception. Our results indicated that stimuli containing highlights led to shorter searching time with the dependence on the light source direction (top lights make searching faster), suggesting that highlights indeed facilitate shape-from-shading processing. To examine how highlight processing interacts with shading processing, we tested unnatural stimuli for which the lighting directions for shading and highlights were inconsistent. The results indicated that unnatural highlights (bright spots) placed in a direction inconsistent with the shading either decrease or do not alter searching time. This suggests that highlights may facilitate, and not suppress, shading processing. With more physically plausible highlights generated from image-based lighting, we also observed facilitation with consistent highlights, but no change with inconsistent highlights. Finally, we examined whether highlights indeed work to facilitate depth perception in a discrimination task. The results showed that correct discrimination of depth increases when highlights are added to shading even when their lighting directions are inconsistent. These results indicate that specular highlights facilitate shading processing, and do not suppress it even when the highlights are placed in a direction inconsistent with shading. The results also elucidate the lighting constraints of the visual system.

Highlights ("bright spots") were used to detect apples in nighttime images.The spatial distribution of the light around the highlights was used to reject false positives.The procedure was tested with two dataset containing over 360 images.The total number of fruit estimated by the procedure was within 10% of visual count. Detection of fruit in tree images has been the focus of numerous studies. Although most studies considered approaches based primarily on color analysis, the major drawback of such approaches is that the fruit apparent color depends not only on variety or physiological stage but also on illumination, which is inherently non-uniform within the canopy, even if artificial lighting is used. In the present work we developed a novel approach to detect apples in nighttime images by analyzing the spatial distribution of the light around highlights ("bright spots"). The approach is based on the observation that, under the artificial illumination used, apples exhibit strong specular reflection so that a small, but very bright, spot is visible on almost all apples. Each of these highlights serves as the center of a region of interest and is the seed of the investigated light patch. This patch is initially very small but its size is increased iteratively by annexing pixels with predefined decreasing gray level intensities. The evolution of the patch geometry is used to determine whether it corresponds to an apple. The approach was tested with two datasets containing over 360 images (close to 13,000 apples) acquired in the same 'Golden Delicious' orchard in July 2012 and August 2013. Twenty images from the 2012 dataset were randomly selected to develop and calibrate the procedure. The results of these 20 images were used to establish a linear relationship between the number of detected objects and the actual number of apples visible in the images (R2~0.75). Applying the calibrated procedure to the remaining images of this dataset led to an estimate of 6739 apples compared to a visual count of 6195 apples (~9% overestimate). Analysis of the 2013 dataset, in which the apparent size of the apples was smaller, required only adjustment of the two parameters related to apple size. Following this adjustment, 12 images were randomly selected to determine the relationship between the number of detected objects and the actual number of apples (R2~0.74). Using this relationship, the estimated number of apples was 6687, compared to the visual count of 6713 fruits.

The process of splitting an image into specular and diffuse components is a fundamental problem in computer vision, because most computer vision algorithms, such as image segmentation and tracking, assume diffuse surfaces, so existence of specular reflection can mislead algorithms to make incorrect decisions. Existing decomposition methods tend to work well for images with low specularity and high chromaticity, but they fail in cases of high intensity specular light and on images with low chromaticity. In this paper, we address the problem of removing high intensity specularity from low chromaticity images (faces). We introduce a new dataset, Spec-Face, comprising face images corrupted with specular lighting and corresponding ground truth diffuse images. We also introduce two deep learning models for specularity removal, Spec-Net and Spec-CGAN. Spec-Net takes an intensity channel as input and produces an output image that is very close to ground truth, while Spec-CGAN takes an RGB image as input and produces a diffuse image very similar to the ground truth RGB image. On Spec-Face, with Spec-Net, we obtain a peak signal-to-noise ratio (PSNR) of 3.979, a local mean squared error (LMSE) of 0.000071, a structural similarity index (SSIM) of 0.899, and a Frechet Inception Distance (FID) of 20.932. With Spec-CGAN, we obtain a PSNR of 3.360, a LMSE of 0.000098, a SSIM of 0.707, and a FID of 31.699. With Spec-Net and Spec-CGAN, it is now feasible to perform specularity removal automatically prior to other critical complex vision processes for real world images, i.e., faces. This will potentially improve the performance of algorithms later in the processing stream, such as face recognition and skin cancer detection.

SummaryRetinal images are produced by interactions between a surface’s 3D shape, material properties, and surrounding light field. In order to recover the 3D geometry of a surface, the visual system must somehow separate aspects of image structure generated by a surface’s shape from structure generated by its material properties or the light field in which it is embedded. Attributing image structure to the wrong physical source would cause the visual system to interpret changes in one physical property (such as reflectance) as changes in another (such as shape). Many previous studies have shown that the visual system does not conflate image structure generated by specular reflectance with 3D shape [1–6], but they did not assess the physical conditions where it would be computationally most difficult to disentangle these different sources of image structure. Here, we show that varying the specular roughness and curvature of surfaces embedded in natural light fields can strongly modulate perceived shape. Despite the complexity of these interactions, we show how an image’s gradient structure mediates its interpretation as a specular reflection or a change in 3D shape. Our findings provide a coherent explanation of when and why specular reflections impact perceived shape and reveal how the static surface properties, simplified light fields, and experimental methods used in previous studies may explain their inconsistent results.

Existing GPU antialiasing techniques, such as MSAA or MLAA, focus on reducing aliasing artifacts along silhouette boundaries or edges in image space. However, they neglect aliasing from shading in case of high-frequency geometric detail. This may lead to a shading aliasing artifact that resembles Bailey's Bead Phenomenon-the degradation of continuous specular highlights to a string of pearls. These types of artifacts are particularly striking for high-quality surfaces. So far, the only way of removing aliasing from shading is by globally supersampling the entire image with a large number of samples. However, globally supersampling the image is slow and significantly increases bandwidth consumption. We propose three adaptive approaches that locally supersample triangles only where necessary on the GPU. Thereby, we efficiently remove artifacts from shading while aliasing along silhouettes is reduced by efficient hardware MSAA.

This paper introduces a novel method for recovering light directions and camera parameters using a single sphere. Traditional methods for estimating light directions using spheres either assume both the radius and center of the sphere being known precisely, or they depend on multiple calibrated views to recover these parameters. In this paper, it will be shown that light directions can be uniquely determined from specular highlights observed in a single view of a sphere without knowing or recovering the exact radius and center of the sphere. Besides, given multiple views of the sphere, it will be shown that the focal length and the relative positions and orientations of the cameras can be determined using the recovered sphere and light directions. Closed form solutions for estimation of light directions and camera poses are presented, and an optimization procedure for estimation of the focal length is introduced. Experimental results on synthetic and real data demonstrates both the accuracy and robustness of the proposed method.

In this paper a robust segmentation and an adaptive SURF descriptor are proposed for iris recognition. Conventional recognition systems extract global features from the iris. However, global features are subject to change for transformation, occlusion and non-uniform illumination. The proposed iris recognition system handles these issues. The input iris image is used to remove specular highlights using an adaptive threshold. Further, the pupil and iris boundaries are localized using a spectrum image based approach. The annular region between the pupil and iris boundaries is transformed into an adaptive strip. The strip is enhanced using a gamma correction approach. Features are extracted from the adaptive strip using Speeded Up Robust Features (SURF). The results obtained using SURF are compared with the existing SIFT descriptor and the proposed approach performs with improved accuracy and reduced computation cost.

The dynamic inspection and reverse engineering of specular freeform surfaces is a challenge so far largely unsolved. Products produced in high quantities in industrial environments are still inspected manually which is labour intensive, expensive, monotonous and subjective. We propose a novel deflectometric hardware setup and methodology to overcome this shortfall. The reflection of a line laser from a moving specular surface is captured on a translucent screen. The resulting complex line provides detailed data about the surface gradients at the intersection of the laser light plane and the surface but cannot create correspondence between points of reflections and surface points. The latter is inherent in a traditional active light scanning setup whose data alone will, for specular surfaces, be subject to high levels of noise. We propose a novel method that by simulation of the reflection fuses the two data sets to allow digitalising moving specular surfaces with the high slope sensitivity of deflectometric methods.

In order to reconstruct a surface which has specular reflection from the grayscale information contained in a single two-dimensional (2D) image, a specular shape from shading (SFS) using the fast marching method is presented. Under assumptions that the camera performs an orthographic projection whose direction is equal to the direction of the light source and that the Phong model is used to depict the reflectance property of the surface, the image irradiance equation for specular surface is derived and it is a first-order nonlinear partial differential equation (PDE). By transforming it into a standard Eiknoal PDE, the fast marching method is applied to compute a solution of the resultant Eiknoal equation. Experiments are tested on both a synthetic vase image and a sphere image and the results demonstrate the effectiveness of the presented specular SFS method.

Iris segmentation plays an important role in an accurate iris recognition system. In less constrained environments where iris images are captured at-a-distance and on-the-move, iris segmentation becomes much more difficult due to the effects of significant variation of eye position and size, eyebrows, eyelashes, glasses and contact lenses, and hair, together with illumination changes and varying focus condition. This paper contributes to robust and accurate iris segmentation in very noisy images. Our main contributions are as follows: (1) we propose a limbic boundary localization algorithm that combines K-Means clustering based on the gray-level co-occurrence histogram and an improved Hough transform, and, in possible failures, a complementary method that uses skin information; the best localization between this and the former is selected. (2) An upper eyelid detection approach is presented, which combines a parabolic integro-differential operator and a RANSAC (RANdom SAmple Consensus)-like technique that utilizes edgels detected by a one-dimensional edge detector. (3) A segmentation approach is presented that exploits various techniques and different image information, following the idea of focus of attention, which progressively detects the eye, localizes the limbic and then pupillary boundaries, locates the eyelids and removes the specular highlight. The proposed method was evaluated in the UBIRIS.v2 testing database by the NICE.I organizing committee. We were ranked #4 among all participants according to the evaluation results.

This paper presents a novel technique for measuring the three-dimensional (3-D) shapes of specular surfaces. In this technique, an LCD monitor plane is used as the diffusive light source. It can be vertically moved to two or more different positions. At each position, sinusoid fringe patterns are displayed on the LCD plane and reflected by the measured specular surface. The distorted fringe patterns are captured with a camera, so that the phase distributions at these positions are measured. From the phases, the locus of the incident ray for each pixel is determined in the least squares sense, and further the three-dimensional shape of the specular surface is reconstructed. In so doing, the restrictions and limitations of the existing techniques in computational complexities, phase ambiguities, and error accumulations are eliminated. The validity of this technique has been demonstrated by experimental results.

To create photorealistic three-dimensional (3D) models of real scenes and objects is a challenging problem that demands advanced knowledge of computer vision and computer graphics. Systems that can reconstruct the 3D model of cultural artefacts have found many applications such as virtual museum and historical archiving. While there are methods for 3D digitization of cultural artefacts with high geometric resolution, there are still limitations in achieving high textural resolution for virtual exhibition. One major problem is that the object surface exhibits specular reflection of illuminated light during the acquisition of surface texture. The shading of the target object does not match to other objects or pictures in the virtual scene. Also, if texture of the object must be composed of multiple images, the mismatch of shading (radiometric difference) among the images can be very prominent. In order to create high visual quality exhibition, the specular reflections must be eliminated and then the virtual scene is relit by a synthetic light source. Most existing methods for the identification and removal of specular reflection component demand special device or rely on information obtained in a single image. In order to reconstruct a complete 3D model, we need to acquire a multi-view image sequence. We therefore propose a new method that is applicable for the separation of diffuse and specular reflection components in multi-view image sequence. Also, our method can tackle the specular reflection across the texture boundary. The image sequence is first normalized by the estimated illumination color. Based on the dichromatic reflection model, the specular chromaticity is replaced by the corresponding diffuse chromaticity, which can always be found in neighboring views with the highlights already faded away. We test the new method in modeling Yixing ceramic teapots. The shape model of the teapot is obtained by a laser scanner. The diffuse image sequence is then used to generate the texture map. We create the virtual scene with the photorealistic 3D teapot model, some synthetic 3D models and still pictures. Interactive exhibition of the artefact is achieved with the control of the mouse and simple keyboard commands. This paper gives an account of the procedures for the creation of interactive virtual exhibition of ceramic artefacts.

We aim to create systems that identify and locate objects by comparing known, 3D shapes to intensity images that they have produced. To do this we focus on verification methods that determine whether a known model in a specific pose is consistent with an image. We build on prior work that has done this successfully for Lambertian objects, to handle a much broader class of shiny objects that produce specular highlights. Our core contribution is a novel method for determining whether a known 3D shape is consistent with the 2D shape of a possible highlight found in an image. We do this using only a qualitative description of highlight formation that is consistent with most models of specular reflection, so no specific knowledge of an object's specular reflectance properties is needed. This allows us to treat non-Lambertian image effects as a positive source of information about object identity, rather than treating them as a potential source of noise. We then show how to integrate information about highlights into a system that also checks the consistency of Lambertian reflectance effects. Also, we show how to model Lambertian reflectance using a reference image, rather than albedos, which can be difficult to measure in shiny objects. We test each aspect of our approach using several different data sets. We demonstrate the potential value of our method of handling specular highlights by building a system that can locate shiny, transparent objects, such as glassware, on table tops. We demonstrate our hybrid methods on pottery, and our use of reference images with face recognition experiments.

Scattered light from a surface contains a huge amount of information including surface slope, roughness, pattern, texture, etc. In this paper, the relationship between the intensity distribution of scattered light from a surface and the 3D surface topography will be investigated. From the geometrical point of view the intensity distribution of scattered light from a surface may be modelled with three components, the specular spike, specular lobe and diffuse lobe. In the situation when light is scattered from a relatively smooth surface whose roughness is much larger than the wavelength of the incident light, the scattered light intensity distribution will consist of only a diffuse lobe and a specular lobe, which are well described by a combination of the Lambertian and Torrance–Sparrow geometrical models. Based on the light scattering phenomenon, in this paper a scanning method capable of measuring the 3D surface topography with 500 mm scanning width, high scanning speed, and micron resolution is presented. The 3D topography measurement is implemented by using a linear photodiode array to measure scattered light intensity distribution at different angles against the surface normal. Then the specular lobe of the scattered light will be extracted at each scanning point to calculate the surface normal distribution. Sequentially, the 3D surface topography is obtained by using gradient integration techniques.

During the past several years, considerable work has been presented on the methods for measuring and modeling the observed reflectance properties of materials. However, most of these works have been done under controlled lighting configurations, and require a constant bidirectional reflectance distribution function (BRDF), which is a measure of the amount of light scattered by a medium from one direction into another, or a homogeneous material over the object regions. This paper discusses a method to estimate the reflectance properties of a textured object under uncontrolled illumination. Assuming that the object has a constant specular reflectance property over its surface, the BRDF can be approximated as a constant specular component and a spatially varying diffuse component. The proposed method will first approximate the illumination distribution of a scene from a set of high dynamic range (HDR) radiance maps of a light probe. Next, a patch on the surface of an object is selected; several HDR intensity maps of the patch taken under different viewing conditions are used to estimate the average specular reflectance parameters of this patch. With the separation of the specular component and the diffuse component of each sample, and using the separated diffuse component, the diffuse reflectance parameters for each point over the surface are next estimated. The test results show that the method is effective for textured objects with constant specular reflectance properties. The method can also be directly applied to objects with piecewise-smooth material, which means that although the surface of an object does not have a constant specular reflectance property; when sub-divided into many smaller meshes, the specular parameter of each mesh can be regarded as uniform.

Despite progress in stereo reconstruction and structure from motion, 3D scene reconstruction from multiple images still faces many difficulties, especially in dealing with occlusions, partial visibility, textureless regions, and specular reflections. Moreover, the problem of recovering a spatially dense 3D representation from many views has not been adequately treated. This document addresses the problems of achieving a dense reconstruction from a sequence of images and analyzing and removing specular highlights. The first part describes an approach for automatically decomposing the scene into a set of spatio-temporal layers (namely EPI-tubes) by analyzing the epipolar plane image (EPI) volume. The key to our approach is to directly exploit the high degree of regularity found in the EPI volume. In contrast to past work on EPI volumes that focused on a sparse set of feature tracks, we develop a complete and dense segmentation of the EPI volume. Two different algorithms are presented to segment the input EPI volume into its component EPI tubes. The second part describes a mathematical characterization of specular reflections within the EPI framework and proposes a novel technique for decomposing a static scene into its diffuse (Lambertian) and specular components. Furthermore, a taxonomy of specularities based on their photometric properties is presented as a guide for designing further separation techniques. The validity of our approach is demonstrated on a number of sequences of complex scenes with large amounts of occlusions and specularity. In particular, we demonstrate object removal and insertion, depth map estimation, and detection and removal of specular highlights.

Two experiments are reported in which observers judged the sign and magnitude of surface curvature from shaded images of an indoor scene. The depicted surfaces were illuminated by an area light in the ceiling or floor, and the illumination was attenuated with distance in a physically correct manner. The displays were presented both with and without cast shadows, specular highlights and surface inter-reflections in all possible combinations. The overall pattern of results revealed a strong perceptual bias to interpret the images as convex rather than concave, and a weaker bias to prefer illumination from above rather than from below, though there were large individual differences in the relative strengths of these biases. For displays that did not contain cast shadows or specular highlights, the accuracy of observers' judgments about the sign of surface curvature was no greater than chance, but performance was significantly improved when those aspects of normal shading were included in the rendering model. An analysis of the apparent depth magnitudes revealed that convex surfaces produce much greater perceived depth than concave surfaces with comparable relief.

A method is described for estimating the refractive index of the material of an object surface with convex shape in natural scenes. The refractive index is an optical constant that is inherent to the surface material. This optical parameter is not only useful for identifying the material of an object surface, but also important for creating realistic images of the object. It is difficult to measure the refractive index of any material in natural scenes, except for transparent materials. In this paper, we consider estimation of the refractive index for relatively smooth surfaces of inhomogeneous dielectric materials by using a calibrated CCD camera. First, we show how the appearance of an image including specular highlights changes with the refractive index. This parameter is then estimated from the camera data of specular highlights of an object surface at different directions of illumination and viewing. The estimation algorithm is based on histogram analysis using the Torrance-Sparrow reflection model. The estimated refractive index is compared with direct measurements using a refractometer. Moreover the feasibility of the proposed method is demonstrated in computer graphics images created by using the estimated reflection parameters.

A region-based approach to nonrigid motion tracking is described. Shape is defined in terms of a deformable triangular mesh that captures object shape plus a color texture map that captures object appearance. Photometric variations are also modeled. Nonrigid shape registration and motion tracking are achieved by posing the problem as an energy-based, robust minimization procedure. The approach provides robustness to occlusions, wrinkles, shadows, and specular highlights. The formulation is tailored to take advantage of texture mapping hard-ware available in many workstations, PCs, and game consoles. This enables nonrigid tracking at speeds approaching video rate.

Binocular disparity is an aspect of natural viewing. This research investigates whether disparity affects surface color perception. Achromatic settings were obtained and compared for two stereograms of a scene with specular reflections, one stereogram with binocular disparity and one without it (cyclopean view). Binocular disparity was found to improve color constancy. Next, the geometry of specular highlights, which is distorted without binocular disparity, was specifically examined. Measurements compared color constancy with specular reflections that were either normal (with stereo disparity) or distorted (cyclopean view of the specularities). No significant change in constancy was found due to the geometrical distortion of specular highlights that occurs without stereo disparity, suggesting that constancy depends on other features of the percept affected by disparity. The results are discussed in terms of illuminant estimation in surface color perception.

Many recent computational models of surface color perception presuppose information about illumination in scenes. The models differ primarily in the physical process each makes use of as a cue to the illuminant. We evaluated whether the human visual system makes use of any of three of the following candidate illuminant cues: (1) specular highlight, (2) full surface specularity [Lee, H. C. (1986). Method for computing the scene-illuminant chromaticity from specular highlights. Journal of the Optical Society of America A, 3(10), 1694–1699; D’Zmura, M., & Lennie, P. (1986). Mechanisms of color constancy. Journal of the Optical Society of America A, 3(10), 1662–1672], and (3) uniform background. Observers viewed simulated scenes binocularly in a computer-controlled Wheatstone stereoscope. All simulated scenes contained a uniform background plane perpendicular to the observer’s line of sight and a small number of specular, colored spheres resting on the uniform background. Scenes were rendered under either standard illuminant D65 or standard illuminant A. Observers adjusted the color of a small, simulated test patch to appear achromatic. In a series of experiments we perturbed the illuminant color signaled by each candidate cue and looked for an influence of the changed cue on achromatic settings. We found that the specular highlight cue had a significant influence, but that the influence was asymmetric: greater when the base illuminant, CIE standard Illuminant A, was perturbed in the direction of Illuminant D65 than vice versa. Neither the full surface specularity cue nor the background cue had any observable influence. The lack of influence of the background cue is likely due to the placement of the test patch in front of the background rather than, as is typical, embedded in the background. © 2001 Elsevier Science Ltd. All rights reserved.

The early detection, classification, and elimination of occlusion and specular highlights which occur frequently in real scenes are of utmost importance. However, most of vergence algorithms which have been proposed in recent years do not take into consideration the effect of occlusion and specular highlights present in real scenes. In this paper, we propose an active exploratory vergence method based on concurrent cross-correlation of multiscale stereo images and an adaptive feedback strategy using weighted normalized cross-correlation to detect and distinguish occlusion and specular highlights concurrently during the vergence process. It not only improves computational efficiency but also leads to the exploration of a better viewing direction and position that overcome the problem created by occlusion and specular highlights. The robustness of the proposed method against occlusion and specular highlights is demonstrated with real scenes.

Visual realism in computer-generated images is an important goal for many computer applications. However, a tradeoff exists between the degree of realism achieved and the time required to generate an image. Thus efficient algorithms for generating realistic images are being actively investigated. Shading is one graphical technique for rendering more realistic images of 3D objects. However, the most widely used algorithm, Gouraud shading, suffers from the Mach band effect, a perceptual phenomenon that reduces realism. Gouraud shading also handles specular reflection poorly. The next most popular algorithm, Phong shading, generally reduces Mach banding and captures specular highlights, though at great computational expense. Bishop and Weimer improved the efficiency of Phong shading, but their algorithm introduces approximation error and is still significantly slower than Gouraud shading. A new algorithm is proposed which takes an innovative approach to shading 3D objects. A smooth intensity surface is generated with a known surface-fitting technique, Powell-Sabin quadratic interpolation. The intensity surface is generated at a modest cost using the Cendes-Wong formulae and forward differencing. The C 1 surface virtually eliminates Mach banding.

We propose a solution to the problem of determining surface material properties from range and intensity data using a simplified version of the Torrance-Sparrow illumination model. The solution uses the photometric stereo method and regularization to invert the model equations at each point on a surface. Assuming a convex surface, one range map, and four or more intensity maps obtained using point light sources, we classify the surface into nonhighlight regions, specular highlight regions, and rank-deficient regions. This classification allows the appropriate solution method to be applied to each region. In nonhighlighted regions we use linear least squares, in highlight regions, nonlinear separable least squares with regularization, and in rank-deficient regions, interpolation. The solution consists of the values of the three parameters of the illumination model at each point on the surface. We believe this technique to be a useful adjunct to recently reported noncontact modeling systems. These systems have been designed to build computer graphics models automatically from real objects by determining surface geometry, surface relief texture, and material properties. Our technique greatly enhances the modeling of material properties. The paper concludes with a number of examples of the method applied to synthetic and real images, and a discussion of possibilities for future systems.

Two shading models that account for, respectively, both diffuse reflection and specular reflection caused by ideal diffuse linear light sources and ideal diffuse polygonal light sources are proposed. In both models, the diffuse reflection component is accurately represented in an analytic form, while the specular reflection component is represented by an integration taking Phong's specular model as the kernel. For linear light sources, a precise recursive formula is derived to calculate the specular reflection component analytically. For polygonal light sources, we adopt Taylor's series to approximate the integration. All the analytic representations of both diffuse and specular terms involve no complex coordinate transformation and the images generated by the shading models are very realistic. Finally, a shadow detection algorithm is presented for the polygonal environment, the precise umbra and penumbra can be produced. Results show great potential of our method.

We show that highlights in images of objects with specularly reflecting surfaces provide significant information about the surfaces which generate them. A brief survey is given of specular reflectance models which have been used in computer vision and graphics. For our work, we adopt the Torrance-Sparrow specular model which, unlike most previous models, considers the underlying physics of specular reflection from rough surfaces. From this model we derive powerful relationships between the properties of a specular feature in an image and local properties of the corresponding surface. We show how this analysis can be used for both prediction and interpretation in a vision system. A shape from specularity system has been implemented to test our approach. The performance of the system is demonstrated by careful experiments with specularly reflecting objects.