Search

Distinguishing between dynamic foreground objects and a mostly static background is a fundamental problem in many computer vision and computer graphics tasks. This paper presents a novel online video background identification method with the assistance of inertial measurement unit (IMU). Based on the fact that the background motion of a video essentially reflects the 3D camera motion, we leverage IMU data to realize a robust camera motion estimation for identifying background feature points by only investigating a few historical frames. We observe that the displacement of the 2D projection of a scene point caused by camera rotation is depth-invariant, and the rotation estimation by using IMU data can be quite accurate. We thus propose to analyze 2D feature points by decomposing the 2D motion into two components: rotation projection and translation projection. In our method, after establishing the 3D camera rotations, we generate the depth-relevant 2D feature point movement induced by the camera 3D translation. Then, by examining the disparity between inter-frame offset and the projection of estimated 3D camera motion, we can identify the background feature points. In the experiments, our online method is able to run at 30FPS with only 1 frame latency and outperforms state-of-the-art background identification and other relevant methods. Our method directly leads to a better camera motion estimation, which is beneficial to many applications like online video stabilization, SLAM, image stitching, etc. [ABSTRACT FROM AUTHOR]

Inferring the scene illumination from a single image is an essential yet challenging task in computer vision and computer graphics. Existing works estimate lighting by regressing representative illumination parameters or generating illumination maps directly. However, these methods often suffer from poor accuracy and generalization. This paper presents Geometric Mover’s Light (GMLight), a lighting estimation framework that employs a regression network and a generative projector for effective illumination estimation. We parameterize illumination scenes in terms of the geometric light distribution, light intensity, ambient term, and auxiliary depth, which can be estimated by a regression network. Inspired by the earth mover’s distance, we design a novel geometric mover’s loss to guide the accurate regression of light distribution parameters. With the estimated light parameters, the generative projector synthesizes panoramic illumination maps with realistic appearance and high-frequency details. Extensive experiments show that GMLight achieves accurate illumination estimation and superior fidelity in relighting for 3D object insertion. The codes are available at https://github.com/fnzhan/Illumination-Estimation [ABSTRACT FROM AUTHOR]

Superpixel segmentation, as a central image processing task, has many applications in computer vision and computer graphics. Boundary alignment and shape compactness are leading indicators to evaluate a superpixel segmentation algorithm. Furthermore, convexity can make superpixels reflect more geometric structures in images and provide a more concise over-segmentation result. In this paper, we consider generating convex and compact superpixels while satisfying the constraints of adhering to the boundary as far as possible. We formulate the new superpixel segmentation into an edge-constrained centroidal power diagram (ECCPD) optimization problem. In the implementation, we optimize the superpixel configurations by repeatedly performing two alternative operations, which include site location updating and weight updating through a weight function defined by image features. Compared with existing superpixel methods, our method can partition an image into fully convex and compact superpixels with better boundary adherence. Extensive experimental results show that our approach outperforms existing superpixel segmentation methods in boundary alignment and compactness for generating convex superpixels. [ABSTRACT FROM AUTHOR]

Recent emerging technologies such AR/VR and HCI are drawing high demand on more comprehensive hand shape understanding, requiring not only 3D hand skeleton pose but also hand shape geometry. In this paper, we propose a deep learning framework to produce 3D hand shape from a single depth image. To address the challenge that capturing ground truth 3D hand shape in the training dataset is non-trivial, we leverage synthetic data to construct a statistical hand shape model and adopt weak supervision from widely accessible hand skeleton pose annotation. To bridge the gap due to the different hand skeleton definitions in the existing public datasets, we propose a joint regression network for hand pose adaptation. To reconstruct the hand shape, we use Chamfer loss between the predicted hand shape and the point cloud from the input depth to learn the shape reconstruction model in a weakly-supervised manner. Experiments demonstrate that our model adapts well to the real data and produces accurate hand shapes that outperform the state-of-the-art methods both qualitatively and quantitatively. [ABSTRACT FROM AUTHOR]

Generating text to have a consistent style with only a few observed highly-stylized text samples is a difficult task for image processing. The text style involving the typography, i.e., font, stroke, color, decoration, effects, etc., should be considered for transfer. In this paper, we propose a novel approach to stylize target text by decoding weighted deep features from only a few referenced samples. The deep features, including content and style features of each referenced text, are extracted from a Convolutional Neural Network (CNN) that is optimized for character recognition. Then, we calculate the similarity scores of the target text and the referenced samples by measuring the distance along the corresponding channels from the content features of the CNN when considering only the content, and assign them as the weights for aggregating the deep features. To enforce the stylized text to be realistic, a discriminative network with adversarial loss is employed. We demonstrate the effectiveness of our network by conducting experiments on three different datasets which have various styles, fonts, languages, etc. Additionally, the coefficients for character style transfer, including the character content, the effect of similarity matrix, the number of referenced characters, the similarity between characters, and performance evaluation by a new protocol are analyzed for better understanding our proposed framework. [ABSTRACT FROM AUTHOR]

Measuring the similarity between the patches in images is a fundamental building block in various tasks. Naturally, the patch size has a major impact on the matching quality and on the consequent application performance. Under the assumption that our patch database is sufficiently sampled, using large patches (e.g., $21 \,\, \times \,\, 21$ ) should be preferred over small ones (e.g., $7 \,\, \times \,\, 7$ ). However, this dense-sampling assumption is rarely true; in most cases, large patches cannot find relevant nearby examples. This phenomenon is a consequence of the curse of dimensionality, stating that the database size should grow exponentially with the patch size to ensure proper matches. This explains the favored choice of small patch size in most applications. Is there a way to keep the simplicity and work with small patches while getting some of the benefits that large patches provide? In this paper, we offer such an approach. We propose to concatenate the regular content of a conventional (small) patch with a compact representation of its (large) surroundings—its context. Therefore, with a minor increase of the dimensions (e.g., with additional ten values to the patch representation), we implicitly/softly describe the information of a large patch. The additional descriptors are computed based on a self-similarity behavior of the patch surrounding. We show that this approach achieves better matches, compared with the use of conventional-size patches, without the need to increase the database-size. Also, the effectiveness of the proposed method is tested on three distinct problems: 1) external natural image denoising; 2) depth image super-resolution; and 3) motion-compensated frame-rate up conversion. [ABSTRACT FROM PUBLISHER]

In this paper, we are concerned with image downsampling using subpixel techniques to achieve superior sharpness for small liquid crystal displays (LCDs). Such a problem exists when a high-resolution image or video is to be displayed on low-resolution display terminals. Limited by the low-resolution display, we have to shrink the image. Signal-processing theory tells us that optimal decimation requires low-pass filtering with a suitable cutoff frequency, followed by downsampling. In doing so, we need to remove many useful image details causing blurring. Subpixel-based downsampling, taking advantage of the fact that each pixel on a color LCD is actually composed of individual red, green, and blue subpixel stripes, can provide apparent higher resolution. In this paper, we use frequency-domain analysis to explain what happens in subpixel-based downsampling and why it is possible to achieve a higher apparent resolution. According to our frequency-domain analysis and observation, the cutoff frequency of the low-pass filter for subpixel-based decimation can be effectively extended beyond the Nyquist frequency using a novel antialiasing filter. Applying the proposed filters to two existing subpixel downsampling schemes called direct subpixel-based downsampling (DSD) and diagonal DSD (DDSD), we obtain two improved schemes, i.e., DSD based on frequency-domain analysis (DSD-FA) and DDSD based on frequency-domain analysis (DDSD-FA). Experimental results verify that the proposed DSD-FA and DDSD-FA can provide superior results, compared with existing subpixel or pixel-based downsampling methods. [ABSTRACT FROM AUTHOR]

This paper proposes a spectral-graph-based algorithm for face image repairing, which can improve the recognition performance on occluded faces. The face completion algorithm proposed in this paper includes three main procedures: 1) sparse representation for partially occluded face classification; 2) image-based data mining; and 3) graph Laplace (GL) for face image completion. The novel part of the proposed framework is GL, as named from graphical models and the Laplace equation, and can achieve a high-quality repairing of damaged or occluded faces. The relationship between the GL and the traditional Poisson equation is proven. We apply our face repairing algorithm to produce completed faces, and use face recognition to evaluate the performance of the algorithm. Experimental results verify the effectiveness of the GL method for occluded face completion. [ABSTRACT FROM PUBLISHER]

Images are conventionally sampled on a rectangular lattice. Thus, traditional image processing is carried out on the rectangular lattice. The hexagonal lattice was proposed more than four decades ago as an alternative method for sampling. Compared with the rectangular lattice, the hexagonal lattice has certain advantages which include that it needs less sampling points; it has better consistent connectivity and higher symmetry; the hexagonal structure is also pertinent to the vision process. In this paper, we investigate the construction of symmetric FIR hexagonal filter banks for multiresolution hexagonal image processing. We obtain block structures of FIR hexagonal filter banks with 3-fold rotational symmetry and 3-fold axial symmetry. These block structures yield families of orthogonal and biorthogonal FIR hexagonal filter banks with 3-fold rotational symmetry and 3-fold axial symmetry. In this paper, we also discuss the construction of orthogonal and biorthogonal FIR filter banks with scaling functions and wavelets having optimal smoothness. In addition, we present a few of such orthogonal and biorthogonal FIR filters banks. [ABSTRACT FROM AUTHOR]

Sparse representations of signals have drawn considerable interest in recent years. The assumption that natural signals, such as images, admit a sparse decomposition over a redundant dictionary leads to efficient algorithms for handling such sources of data. In particular, the design of well adapted dictionaries for images has been a major challenge. The K-SVD has been recently proposed for this task [1] and shown to perform very well for various grayscale image processing tasks. In this paper, we address the problem of learning dictionaries for color images and extend the K-SVD-based grayscale image denoising algorithm that appears in [21. This work puts forward ways for handling nonhomogeneous noise and missing information, paving the way to state-of-the-art results in applications such as color image denoising, demosaicing, and inpainting, as demonstrated in this paper. [ABSTRACT FROM AUTHOR]

In this paper, we put forward a novel fusion framework that mixes together label fields instead of observation data as is usually the case. Our framework takes as input two label fields: a quickly estimated and to-be-refined segmentation map and a spatial region map that exhibits the shape of the main objects of the scene. These two label fields are fused together with a global energy function that is minimized with a deterministic iterative conditional mode algorithm. As explained in the paper, the energy function may implement a pure fusion strategy or a fusion-reaction function. In the latter case, a data-related term is used to make the optimization problem well posed. We believe that the conceptual simplicity, the small number of parameters, the use of a simple and fast deterministic optimizer that admits a natural implementation on a parallel architecture are among the main advantages of our approach. Our fusion framework is adapted to various computer vision applications among which are motion segmentation, motion estimation and occlusion detection. [ABSTRACT FROM AUTHOR]

Snakes, or active contours, have been widely used in image processing applications. Typical roadblocks to consistent performance include limited capture range, noise sensitivity, and poor convergence to concavities. This paper proposes a new external force for active contours, called vector field convolution (VFC), to address these problems. VFC is calculated by convolving the edge map generated from the image with the user-defined vector field kernel. We propose two structures for the magnitude function of the vector field kernel, and we provide an analytical method to estimate the parameter of the magnitude function. Mixed VFC is introduced to alleviate the possible leakage problem caused by choosing inappropriate parameters. We also demonstrate that the standard external force and the gradient vector flow (GVF) external force are special cases of VFC in certain scenarios. Examples and comparisons with GVF are presented in this paper to show the advantages of this innovation, including superior noise robustness, reduced computational cost, and the flexibility of tailoring the force field. [ABSTRACT FROM AUTHOR]

The Fourier transform is one of the most important transformations in image processing. A major component of this influence comes from the ability to implement it efficiently on a digital computer. This paper describes a new methodology to perform a fast Fourier transform (FFT). This methodology emerges from considerations of the natural physical constraints imposed by image capture devices (camera/eye). The novel aspects of the specific FFT method described include: 1) a bit-wise reversal re-grouping operation of the conventional FFT is replaced by the use of lossless image rotation and scaling and 2) the usual arithmetic operations of complex multiplication are replaced with integer addition. The significance of the FFT presented in this paper is introduced by extending a discrete and finite image algebra, named Spiral Honeycomb image Algebra (SHIA), to a continuous version, named SHIAC. [ABSTRACT FROM AUTHOR]

This is Part II of the paper, "Gray-Level Grouping (GLG): an Automatic Method for Optimized Image Contrast Enhancement". Part I of this paper introduced a new automatic contrast enhancement technique: gray-level grouping (GLG). GLG is a general and powerful technique, which can be conveniently applied to a broad variety of low-contrast images and outperforms conventional contrast enhancement techniques. However, the basic GLG method still has limitations and cannot enhance certain classes of low-contrast images well, e.g., images with a noisy background. The basic GLG also cannot fulfill certain special application purposes, e.g., enhancing only part of an image which corresponds to a certain segment of the image histogram. In order to break through these limitations, this paper introduces an extension of the basic GLG algorithm, selective gray-level grouping (SGLG), which groups the histogram components in different segments of the grayscale using different criteria and, hence, is able to enhance different parts of the histogram to various extents. This paper also introduces two new preprocessing methods to eliminate background noise in noisy low-contrast images so that such images can be properly enhanced by the (S)GLG technique. The extension of (S)GLG to color images is also discussed in this paper. SGLG and its variations extend the capability of the basic GLG to a larger variety of low-contrast images, and can fulfill special application requirements. SGLG and its variations not only produce results superior to conventional contrast enhancement techniques, but are also fully automatic under most circumstances, and are applicable to a broad variety of images. [ABSTRACT FROM AUTHOR]

Contrast enhancement has an important role in image processing applications. Conventional contrast enhancement techniques either often fail to produce satisfactory results for a broad variety of low-contrast images, or cannot be automatically applied to different images, because their parameters must be specified manually to produce a satisfactory result for a given image. This paper describes a new automatic method for contrast enhancement. The basic procedure is to first group the histogram components of a low-contrast image into a proper number of bins according to a selected criterion, then redistribute these bins uniformly over the grayscale, and finally ungroup the previously grouped gray-levels. Accordingly, this new technique is named gray-level grouping (GLG). GLG not only produces results superior to conventional contrast enhancement techniques, but is also fully automatic in most circumstances, and is applicable to a broad variety of images. An extension of GLG, selective GLG (SGLG), and its variations will be discussed in Part II of this paper. SGLG selectively groups and ungroups histogram components to achieve specific application purposes, such as eliminating background noise, enhancing a specific segment of the histogram, and so on. The extension of GLG to color images will also be discussed in Part II. [ABSTRACT FROM AUTHOR]

This paper advances a new framework for chromatic filtering of color images. The chromatic content of a color image is encoded in the CIE u' v' chromaticity coordinates whereas the achromatic content is encoded as CIE Y tristimulus value. Within the u' v' chromaticity diagram, colors are added according to the well-known center of gravity law of additive color mixtures, which is generalized here into a nonlinear filtering scheme for processing the two chromatic signals u' and v'. The achromatic channel Y can be processed with traditional filtering schemes, either linear or nonlinear, depending on the specific task at hand. The most interesting characteristics of the new filtering scheme are: 1) the elimination of color smearing effects along edges between bright and dark areas; 2) the possibility of processing chromatic components in a noniterative fashion through linear convolution operations; and 3) the consequent amenability to computationailly efficient implementations with fast Fourier transform. The paper includes several examples with both synthetic and real images where the performance of the new filtering method is compared with that of other color image processing algorithms. [ABSTRACT FROM AUTHOR]

The problem of image classification has aroused considerable research interest in the field of image processing. Traditional methods often convert an image to a vector and then use a vector-based classifier. In this paper, a novel multiple rank regression model (MRR) for matrix data classification is proposed. Unlike traditional vector-based methods, we employ multiple-rank left projecting vectors and right projecting vectors to regress each matrix data set to its label for each category. The convergence behavior, initialization, computational complexity, and parameter determination are also analyzed. Compared with vector-based regression methods, MRR achieves higher accuracy and has lower computational complexity. Compared with traditional supervised tensor-based methods, MRR performs better for matrix data classification. Promising experimental results on face, object, and hand-written digit image classification tasks are provided to show the effectiveness of our method. [ABSTRACT FROM AUTHOR]

This paper describes a new statistical approach for watermarking mesh representations of 3-D graphical objects. A robust digital watermarking method has to mitigate among the requirements of watermark invisibility, robustness, embedding capacity and key security. The proposed method employs a mesh propagation distance metric procedure called the fast marching method (FMM), which defines regions of equal geodesic distance width calculated with respect to a reference location on the mesh. Each of these regions is used for embedding a single bit. The embedding is performed by changing the normalized distribution of local geodesic distances from within each region. Two different embedding methods are used by changing the mean or the variance of geodesic distance distributions. Geodesic distances are slightly modified statistically by displacing the vertices in their existing triangle planes. The vertex displacements, performed according to the FMM, ensure a minimal surface distortion while embedding the watermark code. Robustness to a variety of attacks is shown according to experimental results. [ABSTRACT FROM AUTHOR]

Level set methods have been widely used in image processing and computer vision. In conventional level set formulations, the level set function typically develops irregularities during its evolution, which may cause numerical errors and eventually destroy the stability of the evolution. Therefore, a numerical remedy, called reinitialization, is typically applied to periodically replace the degraded level set function with a signed distance function. However, the practice of reinitialization not only raises serious problems as when and how it should be performed, but also affects numerical accuracy in an undesirable way. This paper proposes a new variational level set formulation in which the regularity of the level set function is intrinsically maintained during the level set evolution. The level set evolution is derived as the gradient flow that minimizes an energy functional with a distance regularization term and an external energy that drives the motion of the zero level set toward desired locations. The distance regularization term is defined with a potential function such that the derived level set evolution has a unique forward-and-backward (FAB) diffusion effect, which is able to maintain a desired shape of the level set function, particularly a signed distance profile near the zero level set. This yields a new type of level set evolution called distance regularized level set evolution (DRLSE). The distance regularization effect eliminates the need for reinitialization and thereby avoids its induced numerical errors. In contrast to complicated implementations of conventional level set formulations, a simpler and more efficient finite difference scheme can be used to implement the DRLSE formulation. DRLSE also allows the use of more general and efficient initialization of the level set function. In its numerical implementation, relatively large time steps can be used in the finite difference scheme to reduce the number of iterations, while ensuring sufficient numerical accuracy. To demonstrate the effectiveness of the DRLSE formulation, we apply it to an edge-based active contour model for image segmentation, and provide a simple narrowband implementation to greatly reduce computational cost. [ABSTRACT FROM AUTHOR]

Copyright protection is a key issue for video sharing over public networks. To protect the video content from unauthorized redistribution, digital fingerprinting is commonly used. To develop an efficient collusion-resistant fingerprinting scheme, it is very important for the system designer to understand how the behavior dynamics of colluders affect the performance of collusion attack. In the literature, little effort has been made to explicitly study the relationship between risk, e.g., the probability of the colluders to be detected, and the distortion of the colluded signal. In this paper, we investigate the risk-distortion relationship for the linear video collusion attack with Gaussian fingerprint. We formulate the optimal linear collusion attack as an optimization problem of finding the optimal collusion parameters to minimize the distortion subject to a risk constraint. By varying the risk constraint and solving the corresponding optimization problem, we can derive the optimal risk-distortion curve. Moreover, based upon the observation that the detector/attacker can each improve the detection/ attack performance with the knowledge of his/her opponent's strategy, we formulate the attack and detection problem as a dynamic mouse and cat game and study the optimal strategies for both the attacker and detector. We show that if the detector uses a fixed detection strategy, the attacker can estimate the detector's strategy and choose the corresponding optimal strategy to attack the fingerprinted video with a small distortion. However, if the detector is powerful, i.e., the detector can always estimate the attacker's strategy, the best strategy for the attacker is the min-max strategy. Finally, we conduct several experiments to verify the proposed risk-distortion model using real video data. [ABSTRACT FROM AUTHOR]

Subspace learning techniques for face recognition have been widely studied in the past three decades. In this paper, we study the problem of general subspace-based face recognition under the scenarios with spatial misalignments and/or image occlusions. For a given subspace derived from training data in a supervised, unsupervised, or semi-supervised manner, the embedding of a new datum and its underlying spatial misalignment parameters are simultaneously inferred by solving a constrained l1 norm optimization problem, which minimizes the l1 error between the misalignment-amended image and the image reconstructed from the given subspace along with its principal complementary subspace. A byproduct of this formulation is the capability to detect the underlying image occlusions. Extensive experiments on spatial misalignment estimation, image occlusion detection, and face recognition with spatial misalignments and/or image occlusions all validate the effectiveness of our proposed general formulation for misalignment-robust face recognition. [ABSTRACT FROM AUTHOR]

Image denoising has been a well studied problem in the field of image processing. Yet researchers continue to focus attention on it to better the current state-of-the-art. Recently proposed methods take different approaches to the problem and yet their denoising performances are comparable. A pertinent question then to ask is whether there is a theoretical limit to denoising performance and, more importantly, are we there yet? As camera manufacturers continue to pack increasing numbers of pixels per unit area, an increase in noise sensitivity manifests itself in the form of a noisier image. We study the performance bounds for the image denoising problem. Ourwork in this paper estimates a lower bound on the mean squared error of the denoised result and compares the performance of current state-of-the-art denoising methods with this bound. We show that despite the phenomenal recent progress in the quality of denoising algorithms, some room for improvement still remains for a wide class of general images, and at certain signal-to-noise levels. Therefore, image denoising is not dead--yet. [ABSTRACT FROM AUTHOR]

In this paper, we propose an approach for the accurate rotation of a digital image using Hermite expansions. This exploits the fact that if a 2-D continuous bandlimited Hermite expansion is rotated, the resulting function can be expressed as a Hermite expansion with the same bandlimit. Furthermore, the Hermite coefficients of the initial 2-D expansion and the rotated expansion are mapped through an invertible linear relationship. Two efficient methods to compute the mapping between Hermite coefficients during rotation are proposed. We also propose a method for connecting the Hermite expansion and a discrete image. Using this method, we can obtain the Hermite expansion from a discrete image and vice versa. Combining these techniques, we propose new methods for the rotation of discrete images. We assess the accuracy of our methods and compare them with an existing FFT-based method implementing three shears. We find that the method proposed here consistently has better accuracy than the FFT-based method. [ABSTRACT FROM AUTHOR]

Centroidal Voronoi tessellations (CVTs) are special Voronoi tessellations whose generators are also the centers of mass (centroids) of the Voronoi regions with respect to a given density function and CVT-based methodologies have been proven to be very useful in many diverse applications in science and engineering. In the context of image processing and its simplest form, CVT-based algorithms reduce to the well-known k-means clustering and are easy to implement. In this paper, we develop edge-weighted centroidal Voronoi tessellation (EWCVT) model for image segmentation and propose some efficient algorithms for its construction. Our EWCVT model can overcome some deficiencies possessed by the basic CVT model; in particular, the new model appropriately combines the image intensity information together with the length of cluster boundaries, and can handle very sophisticated situations. We demonstrate through extensive examples the efficiency, effectiveness, robustness, and flexibility of the proposed method [ABSTRACT FROM AUTHOR]

The logarithmic image processing (LIP) model is a mathematical theory that provides new operations for image processing. The contrast definition has been shown to be consistent with some important physical laws and characteristics of human visual system. In this paper, we establish an information-theoretic interpretation of the contrast definition. We show that it can be expressed as a combination of the relative entropy and Shannon's information content. Based on this new interpretation, we propose an adaptive algorithm for enhancing the contrast and sharpness of noisy images. [ABSTRACT FROM AUTHOR]

In this paper, we study the sampling and the distributed compression of the data acquired by a camera sensor network. The effective design of these sampling and compression schemes requires, however, the understanding of the structure of the acquired data. To this end, we show that the a priori knowledge of the configuration of the camera sensor network can lead to an effective estimation of such structure and to the design of effective distributed compression algorithms. For idealized scenarios, we derive the fundamental performance bounds of a camera sensor network and clarify the connection between sampling and distributed compression. We then present a distributed compression algorithm that takes advantage of the structure of the data and that outperforms independent compression algorithms on real multiview images. [ABSTRACT FROM AUTHOR]

Most full-reference fidelity/quality metrics compare the original image to a distorted image at the same resolution assuming a fixed viewing condition. However, in many applications, such as video streaming, due to the diversity of channel capacities and display devices, the viewing distance and the spatiotemporal resolution of the displayed signal may be adapted in order to optimize the perceived signal quality. For example, at low bitrate coding applications an observer may prefer to reduce the resolution or increase the viewing distance to reduce the visibility of the compression artifacts. The tradeoff between resolution/viewing conditions and visibility of compression artifacts requires new approaches for the evaluation of image quality that account for both image distortions and image size. In order to better understand such tradeoffs, we conducted subjective tests using two representative still image coders, JPEG and JPEG 2000. Our results indicate that an observer would indeed prefer a lower spatial resolution (at a fixed viewing distance) in order to reduce the visibility of the compression artifacts, but not all the way to the point where the artifacts are completely invisible. Moreover, the observer is willing to accept more artifacts as the image size decreases. The subjective test results we report can be used to select viewing conditions for coding applications. They also set the stage for the development of novel fidelity metrics. The focus of this paper is on still images, but it is expected that similar tradeoffs apply to video. [ABSTRACT FROM AUTHOR]

The observation has been made by Aujol and Gilboa that the cartoon and texture components of the decomposition of an image should not be correlated, as they are generated from independent processes. They use this observation in order to choose an optimal fidelity parameter λ for the decomposition process. However, this determination can be quite inefficient since a wide range of parameters λ must be searched through before an estimated optimal parameter can be found. In the present paper, we take a different approach, in which the cartoon and texture components are explicitly decorrelated by adding a decorrelation term to the energy functional of the decomposition model of Osher, Solé, and Vese (the OSV model). Decomposition results of improved quality over those from the OSV model are obtained, as quantified by a series of new decomposition quality measures, with cartoon and texture information better separated into their respective components. A new derivation of the OSV model is developed which maintains the texture subcomponents g1 and g2 so that discrimination results similar to those from other decomposition models (e.g., from the model of Vese and Osher and Improved Edge Segregation) may be obtained. This derivation is extended to the proposed model, for which discrimination results are obtained in a substantially smaller number of iterations. [ABSTRACT FROM AUTHOR]

This paper presents a novel approach for object skeleton family extraction. The introduced technique utilizes a 2-D physics-based deformable model that parameterizes the objects shape. Deformation equations are solved exploiting modal analysis, and proportional to model physical characteristics, a different skeleton is produced every time, generating, in this way, a family of skeletons. The theoretical properties and the experiments presented demonstrate that obtained skeletons match to hand-labeled skeletons provided by human subjects, even in the presence of significant noise and shape variations, cuts and tears, and have the same topology as the original skeletons. In particular, the proposed approach produces no spurious branches without the need of any known skeleton pruning method. [ABSTRACT FROM AUTHOR]

This paper considers the problem of high-dynamic-range (HDR) image capture using low-dynamic-range (LDR) cameras. We present three different minimal-bracketing algorithms for computing minimum-sized exposure sets bracketing of HDR scenes. Each algorithm is applicable to a different HDR-imaging scenario depending on the amount of target-scene-irradiance information and real-time image processing available at the time of image acquisition. We prove the optimality of each algorithm with respect to its ability to obtain a theoretically minimum-size bracketing set of exposures. We also provide closed-form expressions for computing minimal-bracketing exposure sets for two common types of HDR-imaging systems, those with geometrically varying and arithmetically varying exposure settings. We experimentally demonstrate the advantages of the proposed methods by capturing and processing multiple HDR scenes using minimal-bracketing and 1-stop bracketing methods. The results show that minimal-bracketing can be used to produce high-quality HDR images, while requiring only one third as many LDR images be acquired compared to 1-stop bracketing. We also perform a detailed SNR analysis that quantifies the tradeoff between signal-to-noise ratio and image-bracketing-set size. [ABSTRACT FROM AUTHOR]

Active models have been widely used in image processing applications. A crucial stage that affects the ultimate active model performance is initialization. This paper proposes a novel automatic initialization approach for parametric active models in both 2-D and 3-D. The PIG initialization method exploits a novel technique that essentially estimates the external energy field from the external force field and determines the most likely initial segmentation. Examples and comparisons with two state-of-the-art automatic initialization methods are presented to illustrate the advantages of this innovation, including the ability to choose the number of active models deployed, rapid convergence, accommodation of broken edges, superior noise robustness, and segmentation accuracy. [ABSTRACT FROM AUTHOR]

Image-based rendering has been successfully used to display 3-D objects for many applications. A well-known example is the object movie, which is an image-based 3-D object composed of a collection of 2-D images taken from many different viewpoints of a 3-D object. In order to integrate image-based 3-D objects into a chosen scene (e.g., a panorama), one has to meet a hard challenge—to efficiently and effectively remove the background from the foreground object. This problem is referred to as multiview images (MVIs) segmentation. Another task requires MVI segmentation is image-based 3-D reconstruction using multiview images. In this paper, we propose a new method for segmenting MVI, which integrates some useful algorithms, including the well-known graph-cut image segmentation and volumetric graph-cut. The main idea is to incorporate the shape prior into the image segmentation process. The shape prior introduced into every image of the MVI is extracted from the 3-D model reconstructed by using the volumetric graph cuts algorithm. Here, the constraint obtained from the discrete medial axis is adopted to improve the reconstruction algorithm. The proposed MVI segmentation process requires only a small amount of user intervention, which is to select a subset of acceptable segmentations of the MVI after the initial segmentation process. According to our experiments, the proposed method can provide not only good MVI segmentation, but also provide acceptable 3-D reconstructed models for certain less-demanding applications. [ABSTRACT FROM AUTHOR]

This paper introduces a new edge-grouping method to detect perceptually salient structures in noisy images. Specifically, we define a new grouping cost function in a ratio form, where the numerator measures the boundary proximity of the resulting structure and the denominator measures the area of the resulting structure. This area term introduces a preference towards detecting larger-size structures and, therefore, makes the resulting edge grouping more robust to image noise. To find the optimal edge grouping with the minimum grouping cost, we develop a special graph model with two different kinds of edges and then reduce the grouping problem to finding a special kind of cycle in this graph with a minimum cost in ratio form. This optimal cycle-finding problem can be solved in polynomial time by a previously developed graph algorithm. We implement this edge-grouping method, test it on both synthetic data and real images, and compare its performance against several available edge-grouping and edge-linking methods. Furthermore, we discuss several extensions of the proposed method, including the incorporation of the well-known grouping cues of continuity and intensity homogeneity, introducing a factor to balance the contributions from the boundary and region information, and the prevention of detecting self-intersecting boundaries. [ABSTRACT FROM AUTHOR]

Most digital still cameras acquire imagery with a color filter array (CFA), sampling only one color value for each pixel and interpolating the other two color values afterwards. The interpolation process is commonly known as demosaicking. In general, a good demosaicking method should preserve the high-frequency information of imagery as much as possible, since such information is essential for image visual quality. We discuss in this paper two key observations for preserving high-frequency information in CFA demosaicking: 1) the high frequencies are similar across three color components, and 2) the high frequencies along the horizontal and vertical axes are essential for image quality. Our frequency analysis of CFA samples indicates that filtering a CFA image can better preserve high frequencies than filtering each color component separately. This motivates us to design an efficient filter for estimating the luminance at green pixels of the CFA image and devise an adaptive filtering approach to estimating the luminance at red and blue pixels. Experimental results on simulated CFA images, as well as raw CFA data, verify that the proposed method outperforms the existing state-of-the-art methods both visually and in terms of peak signal-to-noise ratio, at a notably lower computational cost. [ABSTRACT FROM AUTHOR]

Circular features are commonly sought in digital image processing. The subspace-based line detection (SLIDE) method proposed to estimate the center and the radius of a single circle. In this paper, we introduce a novel method for estimating several radii while extending the circle estimation to retrieve circular-like distorted contours. Particularly, we develop and validate a new model for virtual signal generation by simulating a circular antenna. The circle center is estimated by the SLIDE method. A variable speed propagation scheme toward the circular antenna yields a linear phase signal. Therefore, a high-resolution method provides the radius. Either the gradient method or the more robust combination of dividing rectangles and spline interpolation can extend this method extend this method for free form object segmentation. The retrieval of multiple non concentric circles and rotated ellipses is also considered. To evaluate the performance of the proposed methods, we compare them with a least-squares method, Hough transform, and gradient vector flow. We apply the proposed method to hand-made images while considering some real-world images. [ABSTRACT FROM AUTHOR]

We present a technique for constructing a ‘clean’ texture map of a partially occluded building facade from a series of images taken from a moving camera. Building regions blocked by trees, signs, people, and other foreground objects in a minority of views can be recovered via temporal median filtering on a registered image mosaic of the planar facade. However, when such areas are occluded in the majority of camera views, appearance information from other visible portions of the facade provides a critical cue to correctly complete the mosaic. In this paper, we apply a robust measure of spread to infer whether a particular mosaic pixel is occluded in a majority of views, and introduce a novel spatiotemporal timeline-based inpainting algorithm that uses appearance and motion cues in order to fill the texture map in majority-occluded regions. We describe methods for automatically training appearance-based classifiers from a coarse motion-based segmentation to efficiently recognize foreground and background patches in static imagery. Results of recovered building facades are shown for various sequences. [ABSTRACT FROM AUTHOR]

Histogram specification (equalization) is an important tool for tasks such as image enhancement and normalization. Although this problem has exact solution for continuous images, in the case of digital images, it is ill posed. Recently, an exact pixel ordering method based solely on local image intensity was proposed, yet still with some limitations, especially since it ignores the important image edge information. In this paper, we present a wavelet-based method that simultaneously achieves the exact histogram specification and good image enhancement performance. It does so through a carefully designed strict pixel ordering process, during which the wavelet coefficients are fine tuned for the image enhancement purpose. Compared to previous work, this approach takes into account not only local mean intensity values, but also local edge information. Other advantages include fast pixel ordering, good statistical models, and better image enhancement performance. Experimental results and comparison with state-of-the-art methods are presented. [ABSTRACT FROM AUTHOR]

Single sensor digital color still/video cameras capture images using a color filter array (CFA) and require color interpolation (demosaicking) to reconstruct full color images. The color reproduction has to combat sensor noises which are channel dependent. If untreated in demosaicking, sensor noises can cause color artifacts that are hard to remove later by a separate denoising process, because the demosaicking process complicates the noise characteristics by blending noises of different color channels. This paper presents a joint demosaicking-denoising approach to overcome this difficulty. The color image is restored from noisy mosaic data in two steps. First, the difference signals of color channels are estimated by linear minimum mean square-error estimation. This process exploits both spectral and spatial correlations to simultaneously suppress sensor noise and interpolation error. With the estimated difference signals, the full resolution green channel is recovered. The second step involves in a wavelet-based denoising process to remove the CFA channel-dependent noises from the reconstructed green channel. The red and blue channels are subsequently recovered. Simulated and real CFA mosaic data are used to evaluate the performance of the proposed joint demosaicking-denoising scheme and compare it with many recently developed sophisticated demosaicking and denoising schemes. [ABSTRACT FROM AUTHOR]

It is well known that the performance of context-based image processing systems can be improved by allowing the processor (e.g., an encoder or a denoiser) a delay of several samples before making a processing decision. Often, however, for such systems, traditional delayed-decision algorithms can become computationally prohibitive due to the growth in the size of the space of possible solutions. In this paper, we propose a reduced-complexity, one-pass, delayed-decision algorithm that systematically reduces the size of the search space, while also preserving its structure. In particular, we apply the proposed algorithm to two examples of adaptive context-based image processing systems, an image coding system that employs a context-based entropy coder, and a spatially adaptive image-denoising system. For these two types of widely used systems, we show that the proposed delayed-decision search algorithm outperforms instantaneous-decision algorithms with only a small increase in complexity. We also show that the performance of the proposed algorithm is better than that of other, higher complexity, delayed-decision algorithms. [ABSTRACT FROM AUTHOR]

Abstract-In this paper, we propose an algorithm for lossless conversion of data between Cartesian and polar coordinates, when the data is sampled from a 2-D real-valued function (a mapping: R² ↦ R) expressed as a particular kind of truncated expansion. We use Laguerre functions and the Fourier basis for the polar coordinate expression. Hermite functions are used for the Cartesian coordinate expression. A finite number of coefficients for the truncated expansion specifies the function in each coordinate system. We derive the relationship between the coefficients for the two coordinate systems. Based on this relationship, we propose an algorithm for lossless conversion between the two coordinate systems. Resampling can be used to evaluate a truncated expansion on the complementary coordinate system without computing a new set of coefficients. The resampled data is used to compute the new set of coefficients to avoid the numerical instability associated with direct conversion of the coefficients. In order to apply our algorithm to discrete image data, we propose a method to optimally fit a truncated expression to a given image. We also quantify the error that this filtering process can produce. Finally the algorithm is applied to solve the polar-Cartesian interpolation problem. [ABSTRACT FROM AUTHOR]

It has been well established that critically sampled boundary pre-/postfiltering operators can improve the coding efficiency and mitigate blocking artifacts in traditional discrete cosine transform-based block coders at low bit rates. In these systems, both the prefilter and the postfilter are square matrices. This paper proposes to use undersampled boundary pre- and postfiltering modules, where the pre-/postfilters are rectangular matrices. Specifically, the prefilter is a ‘fat’ matrix, while the post-filter is a ‘tall’ one. In this way, the size of the prefiltered image is smaller than that of the original input image, which leads to improved compression performance and reduced computational complexities at low bit rates. The design and VLSI-friendly implementation of the undersampled pre-/postfilters are derived. Their relations to lapped transforms and filter banks are also presented. Two design examples are also included to demonstrate the validity of the theory. Furthermore, image coding results indicate that the proposed undersampled pre-/postfiltering systems yield excellent and stable performance in low bit-rate image coding. [ABSTRACT FROM AUTHOR]

This paper proposes a new bimodal biometric system using feature-level fusion of hand shape and palm texture. The proposed combination is of significance since both the palmprint and hand-shape images are proposed to be extracted from the single hand image acquired from a digital camera. Several new hand-shape features that can be used to represent the hand shape and improve the performance are investigated. The new approach for paimprint recognition using discrete cosine transform coefficients, which can be directly obtained from the camera hardware, is demonstrated. None of the prior work on hand-shape or palm-print recognition has given any attention on the critical issue of feature selection. Our experimental results demonstrate that while majority of palmprint or hand-shape features are useful in predicting the subjects identity, only a small subset of these features are necessary in practice for building an accurate model for identification. The comparison and combination of proposed features is evaluated on the diverse classification schemes; naive Bayes (normal, estimated, multinomial), decision trees (C4.5, LMT), k-NN, SVM, and FFN. Although more work remains to be done, our results to date indicate that the combination of selected hand-shape and palmprint features constitutes a promising addition to the biometrics-based personal recognition systems. [ABSTRACT FROM AUTHOR]

Simple point detection is an important task for several problems in discrete geometry, such as topology preserving thinning in image processing to compute discrete skeletons. In this paper, the approach to simple point detection is based on techniques from cubical homology, a framework ideally suited for problems in image processing. A (d-dimensional) unitary cube (for a d-dimensional digital image) is associated with every discrete picture element, instead of a point in ϵd (the d-dimensional Euclidean space) as has been done previously. A simple point in this setting then refers to the removal of a unitary cube without changing the topology of the cubical complex induced by the digital image. The main result is a characterization of a simple point p (i.e., simple unitary cube) in terms of the homology groups of the (3d - 1) neighborhood of p for arbitrary, finite dimensions [ABSTRACT FROM AUTHOR]

This paper introduces edge-forming schemes for image zooming of color images by general magnification factors. In order to remove/reduce artifacts arising in image interpolation, such as image blur and the checkerboard effect, an edge-forming method is suggested to be applied as a postprocess of standard interpolation methods. The method is based on nonconvex non-linear partial differential equations. The equations are carefully discretized, incorporating numerical schemes of anisotropic diffusion, to be able to form reliable edges satisfactorily. The alternating direction implicit (ADI) method is employed for an efficient simulation of the model. It has been numerically verified that the resulting algorithm can form clear edges in 2 to 3 AD! iterations. Various results are given to show th eeffectiveness and reliability of the algorithm. [ABSTRACT FROM AUTHOR]

This paper deals with the problem of badly posed image classification. Although underestimated in practice, bad-posedness is likely to affect many real-world image classification tasks, where reference samples are difficult to collect (e.g., in remote sensing (RS) image mapping) and/or spatial autocorrelation is relevant. In an image classification context affected by a lack of reference samples, an original inductive learning multiscale image classifier, termed multiscale semisupervised expectation maximization (MSEM), is proposed. The rationale behind MSEM is to combine useful complementary properties of two alternative data mapping procedures recently published outside of image processing literature, namely, the multiscale modified Pappas adaptive clustering (MPAC) algorithm and the sample-based semisupervised expectation maximization (SEM) classifier. To demonstrate its potential utility, MSEM is compared against nonstandard classifiers, such as MPAC, SEM and the single-scale contextual SEM (CSEM) classifier, besides against well-known standard classifiers in two RS image classification problems featuring few reference samples and modestly useful texture information. These experiments yield weak (subjective) but numerous quantitative map quality indexes that are consistent with both theoretical considerations and qualitative evaluations by expert photointerpreters. According to these quantitative results, MSEM is competitive in terms of overall image mapping performance at the cost of a computational overhead three to six times superior to that of its most interesting rival, SEM. More in general, our experiments confirm that, even if they rely on heavy class-conditional normal distribution assumptions that may not be true in many real-world problems (e.g., in highly textured images), semisupervised classifiers based on the iterative expectation maximization Gaussian mixture model solution can be very powerful in practice when: 1) there is a lack of reference samples with respect to the problem/model complexity and 2) texture information is considered negligible (i.e., a piecewise constant image model holds). [ABSTRACT FROM AUTHOR]

A new methodology for fingerprinting and water- marking three-dimensional (3-D) graphical objects is proposed in this paper. The 3-D graphical objects are described by means of polygonal meshes. The information to be embedded is provided as a binary code. A watermarking methodology has two stages: embedding and detecting the information that has been embedded in the given media. The information is embedded by means of local geometrical perturbations while maintaining the local connectivity. A neighborhood localized measure is used for selecting appropriate vertices for watermarking. A study is undertaken in order to verify the suitability of this measure for selecting vertices from regions where geometrical perturbations are less perceptible. Two different watermarking algorithms, that do not require the original 3-D graphical object in the detection stage, are proposed. The two algorithms differ with respect to the type of constraint to be embedded in the local structure: by using parallel planes and bounding ellipsoids, respectively. The information capacity of various 3-D meshes is analyzed when using the proposed 3-D watermarking algorithms. The robustness of the 3-D watermarking algorithms is tested to noise perturbation and to object cropping. [ABSTRACT FROM AUTHOR]

This paper studies video halftoning that renders a dig- ital video sequence onto display devices, which have limited intensity resolutions and color palettes, by trading the spatiotemporal resolution for enhanced intensity/color resolution. This trade is needed when a continuous tone video is not necessary or not practical for video display, transmission, and storage. In particular, the quantization error of a pixel is diffused to its spatiotemporal neighbors by separable one-dimensional temporal and two- dimensional spatial error diffusions. Motion-adaptive gain control is employed to enhance the temporal consistency of the visual patterns by minimizing the flickering artifacts. Experimental results of halftone and colortone videos are demonstrated and evaluated with various halftoning techniques. [ABSTRACT FROM AUTHOR]

In this contribution, image processing techniques are investigated to robustly segment faint striation patterns from mainly isotropic background areas. Whereas well-known procedures to determine local anisotropy can be applied directly to single images, this paper describes an alternative strategy that uses an illumination series. The series is obtained with a spot illumination whose azimuth is varied systematically. The technique is based on the characteristics of the local contrast with respect to the azimuth of the illumination: Whereas pronounced maxima of the local contrast can be observed when the striae are illuminated perpendicularly, an isotropic background texture shows less distinct maxima for random azimuth angles. The qualification of the approach is demonstrated on the segmentation of faint tool marks in forensic science. Experimental results show that the methodology ensures the correct segmentation of such marks. [ABSTRACT FROM AUTHOR]

Self-similar random fields are of interest in various areas of image processing since they fit certain types of natural pat- terns and textures. Current treatments of self-similarity in continuous two-dimensional (2-D) space use a definition that is a direct extension of the one-dimensional definition, which requires invariance of the statistics of a random process to time scaling. Current discrete-space 2-D approaches do not consider scaling, but, instead, are based on ad hoc formulations, such as digitizing continuous random fields. In this paper, we show that the current statistical self-similarity definition in continuous space is restrictive and provide an alternative, more general definition. We also provide a formalism for discrete-space statistical self-similarity that relies on a new scaling operator for discrete images. Within the new framework, it is possible to synthesize a wider class of discrete-space self-similar random fields and texture images. [ABSTRACT FROM AUTHOR]

The proliferation of digital information in our society has enticed a lot of research into data-embedding techniques that add information to digital content, like images, audio, and video. In this paper, we investigate high-capacity lossless data-embedding methods that allow one to embed large amounts of data into digital images (or video) in such a way that the original image can be reconstructed from the watermarked image. We present two new techniques: one based on least significant bit prediction and Sweldens' lifting scheme and another that is an improvement of Tian's technique of difference expansion. The new techniques are then compared with various existing embedding methods by looking at capacity-distortion behavior and capacity control. [ABSTRACT FROM AUTHOR]