Search

Textures contain a wealth of image information and are widely used in various fields such as computer graphics and computer vision. With the development of machine learning, the texture synthesis and generation have been greatly improved. As a very common element in everyday life, wallpapers contain a wealth of texture information, making it difficult to annotate with a simple single label. Moreover, wallpaper designers spend significant time to create different styles of wallpaper. For this purpose, this paper proposes to describe wallpaper texture images by using multi-label semantics. Based on these labels and generative adversarial networks, we present a framework for perception driven wallpaper texture generation and style transfer. In this framework, a perceptual model is trained to recognize whether the wallpapers produced by the generator network are sufficiently realistic and have the attribute designated by given perceptual description; these multi-label semantic attributes are treated as condition variables to generate wallpaper images. The generated wallpaper images can be converted to those with well-known artist styles using CycleGAN. Finally, using the aesthetic evaluation method, the generated wallpaper images are quantitatively measured. The experimental results demonstrate that the proposed method can generate wallpaper textures conforming to human aesthetics and have artistic characteristics., IEEE Transactions on Circuits and Systems for Video Technology

Dynamic texture refers to the content in video sequences that is characterized by spatial repetition and temporal variation, such as swaying foliage and flowing water. It is a great challenge to compress the dynamic textures efficiently in the current prediction/transform hybrid video coding framework. However, these textures have little information for machine vision, and human visual perception is less sensitive to the textures than to the structures. Thus, we propose a spatiotemporal generative adversarial network (GAN) based dynamic texture synthesis method for surveillance video coding. We detect and remove the dynamic texture content at encoder side, which is irrelevant to machine vision. We generate the dynamic texture content using the proposed GAN at decoder side, so that the reconstructed videos can be observed by human without deteriorating perceptual quality. Specifically, we design a GAN network to synthesize dynamic textures by exploiting the correlation between spatial and temporal neighbors; we present a surveillance video coding scheme with the dynamic texture detection/synthesis method; we build a high-quality dynamic texture dataset, and we collect a dynamic texture testing dataset that goes beyond the existing video coding test datasets by focusing on surveillance scenes. The proposed video coding scheme has been implemented on top of the High Efficiency Video Coding (HEVC) reference software. Experiments have been conducted to evaluate the quantitative and qualitative performance of the proposed coding scheme. Our method achieves 7.4% and 7.6% bit-rate savings in low-delay-B and low-delay-P settings, respectively, at similar visual quality levels in comparison with HEVC.

We introduce an automatic texture synthesis-based framework to convert an arbitrary input image into embroidery style art for garment design and online display. Given an input image and some reference textures, we first extract key embroidery regions from the input image using image segmentation. Each segmented region is single-colored and labeled with a stitch style automatically. We then fill these regions with embroidery reference textures via a stitch-style-based texture synthesis method. For each region, our approach maintains color similarity before and after synthesis, along with stitch style consistency. Compared to existing approaches, our method is able to generate digital embroidery patterns with faithful details automatically. Moreover, it can accept diverse input images effectively, enabling a fast preview of the embroidery patterns synthesized on digital garments interactively, and therefore accelerating the workflow from design to production. We validate our method through extensive experimentation and comparison.

Random-needle embroidery is a kind of traditional Chinese embroidery where artists reproduce the original color image using multilayered intersecting stitches and overlapped deliberately. In other words, artists embroider intersecting stitches layer by layer to depict true color subjects. To create this embroidery art, the major challenge lies on evenly distribution of intersecting stitches to prevent the canvas in embroidery areas from being exposed. Meanwhile, intersecting stitches in each layer should be distributed as evenly as possible so that each layer of intersecting stitch can be quantified as a segment color to synthesize the target segment color. To tackle these challenges, we introduce local texture growth mechanism from example-based discrete element texture synthesis to the application of random-needle embroidery art stylization and propose a new local growth where a uniformity metric is designed to keep intersecting stitches nearly evenly distributed in each layer. In addition, we adopt a color dithering model to specify a color for each layer of monochromatic intersecting stitches at microscopic scale. We apply our method on a rich variety of images, and visually appealing random-needle embroidery images can be obtained in all cases.

Encountered-type haptic rendering provides realistic, free-to-touch, and move-and-collide haptic sensation to a user. However, inducing haptic-texture sensation without complicated tactile actuators is challenging for encountered-type haptic rendering. In this article, we propose a novel texture synthesizing method for an encountered-type haptic display using spatial and temporal encoding of roughness, which provides both active and passive touch sensation requiring no complicated tactile actuation. Focused on macro-scale roughness perception, we geometrically model the textured surface with a grid of hemiellipsoidal bumps, which can provide a variety of perceived roughness as the user explores the surface with one's bare hand. Our texture synthesis method is based on two important hypotheses. First, we assume that perceptual roughness can be spatially encoded along the radial direction of a textured surface with hemiellipsoidal bumps. Second, perceptual roughness temporally varies with the relative velocity of a scanning human hand with respect to the surface. To validate these hypotheses on our spatiotemporal encoding method, we implemented an encountered-type haptic texture rendering system using an off-the-shelf collaborative robot that can also track the user's hand using IR sensors. We performed psychophysical user tests with 25 participants and verified the main effects of spatiotemporal encoding of a textured model on the user's roughness perception. Our empirical experiments imply that the users perceive a more rough texture as the surface orientation or the relative hand motion increases. Based on these findings, we show that our visuo-haptic system can synthesize an appropriate level of roughness corresponding to diverse visual textures by suitably choosing encoding values.

This paper addresses the problem of creating camouflage images. Such images typically contain one or more hidden objects embedded into a background image, so that viewers are required to consciously focus to discover them. Previous methods basically rely on hand-crafted features and texture synthesis to create camouflage images. However, due to lack of reliable understanding of what essentially makes an object recognizable, they typically result in either complete standout or complete invisible hidden objects. Moreover, they may fail to produce seamless and natural images because of the sensitivity to appearance differences. To overcome these limitations, we present a novel neural style transfer approach that adopts the visual perception mechanism to create camouflage images, which allows us to hide objects more effectively while producing natural-looking results. In particular, we design an attention-aware camouflage loss to adaptively mask out information that make the hidden objects visually standout, and also leave subtle yet enough feature clues for viewers to perceive the hidden objects. To remove the appearance discontinuities between the hidden objects and the background, we formulate a naturalness regularization to constrain the hidden objects to maintain the manifold structure of the covered background. Extensive experiments show the advantages of our approach over existing camouflage methods and state-of-the-art neural style transfer algorithms.

To peripheral vision, a pair of physically different images can look the same. Such pairs are metamers relative to each other, just as physically-different spectra of light are perceived as the same color. We propose a real-time method to compute such ventral metamers for foveated rendering where, in particular for near-eye displays, the largest part of the framebuffer maps to the periphery. This improves in quality over state-of-the-art foveation methods which blur the periphery. Work in Vision Science has established how peripheral stimuli are ventral metamers if their statistics are similar. Existing methods, however, require a costly optimization process to find such metamers. To this end, we propose a novel type of statistics particularly well-suited for practical real-time rendering: smooth moments of steerable filter responses. These can be extracted from images in time constant in the number of pixels and in parallel over all pixels using a GPU. Further, we show that they can be compressed effectively and transmitted at low bandwidth. Finally, computing realizations of those statistics can again be performed in constant time and in parallel. This enables a new level of quality for foveated applications such as such as remote rendering, level-of-detail and Monte-Carlo denoising. In a user study, we finally show how human task performance increases and foveation artifacts are less suspicious, when using our method compared to common blurring.

Face sketch synthesis shows great applications in a lot of fields such as online entertainment and suspects identification. Existing face sketch synthesis methods learn the patch-wise sketch style from the training dataset containing photo-sketch pairs. These methods manipulate the whole process directly in the field of RGB space, which unavoidably results in unsmooth noises at patch boundaries. If denoising methods are used, the sketch edges would be blurred and face structures could not be restored. Recent researches of feature maps, which are the outputs of a certain neural network layer, have achieved great success in texture synthesis and artistic image generation. In this paper, we reformulate the face sketch synthesis problem into a neural network feature maps based optimization task. Our results accurately capture the sketch drawing style and make full use of the whole stylistic information hidden in the training dataset. Unlike former feature map based methods, we utilize the Enhanced 3D PatchMatch and cross-layer cost aggregation methods to obtain the target feature maps for the final results. Multiple experiments have shown that our approach imitates hand-drawn sketch style vividly, and has high-quality visual effects on CUHK, AR, XM2VTS and CUFSF face sketch datasets.

International audience; This paper describes a novel approach for on demand volumetric texture synthesis based on a deep learning framework that allows for the generation of high quality 3D data at interactive rates. Based on a few example images of textures, a generative network is trained to synthesize coherent portions of solid textures of arbitrary sizes that reproduce the visual characteristics of the examples along some directions. To cope with memory limitations and computation complexity that are inherent to both high resolution and 3D processing on the GPU, only 2D textures referred to as "slices" are generated during the training stage. These synthetic textures are compared to exemplar images via a perceptual loss function based on a pre-trained deep network. The proposed network is very light (less than 100k parameters), therefore it only requires sustainable training (i.e. few hours) and is capable of very fast generation (around a second for 256^3 voxels) on a single GPU. Integrated with a spatially seeded PRNG the proposed generator network directly returns an RGB value given a set of 3D coordinates. The synthesized volumes have good visual results that are at least equivalent to the state-of-the-art patch based approaches. They are naturally seamlessly tileable and can be fully generated in parallel.

From angling smiles to duck faces, all kinds of facial expressions can be seen in selfies, portraits, and Internet pictures. These photos are taken from various camera types, and under a vast range of angles and lighting conditions. We present a deep learning framework that can fully normalize unconstrained face images, i.e., remove perspective distortions, relight to an evenly lit environment, and predict a frontal and neutral face. Our method can produce a high resolution image while preserving important facial details and the likeness of the subject, along with the original background. We divide this ill-posed problem into three consecutive normalization steps, each using a different generative adversarial network that acts as an image generator. Perspective distortion removal is performed using a dense flow field predictor. A uniformly illuminated face is obtained using a lighting translation network, and the facial expression is neutralized using a generalized facial expression synthesis framework combined with a regression network based on deep features for facial recognition. We introduce new data representations for conditional inference, as well as training methods for supervised learning to ensure that different expressions of the same person can yield to not only a plausible but also a similar neutral face. We demonstrate our results on a wide range of challenging images collected in the wild. Key applications of our method range from robust image-based 3D avatar creation, portrait manipulation, to facial enhancement and reconstruction tasks for crime investigation. We also found through an extensive user study, that our normalization results can be hardly distinguished from ground truth ones if the person is not familiar.

We show how to overcome the single weakness of an existing fully automatic system for acquisition of spatially varying optical material behavior of real object surfaces. While the expression of spatially varying material behavior with spherical dependence on incoming light as a 4D texture (an ABTF material model) allows flexible mapping onto arbitrary 3D geometry, with photo-realistic rendering and interaction in real time, this very method of texture-like representation exposes it to common problems of texturing, striking in two disadvantages. Firstly, non-seamless textures create visible artifacts at boundaries. Secondly, even a perfectly seamless texture causes repetition artifacts due to their organised placement in large numbers over a 3D surface. We have solved both problems through our novel texture synthesis method that generates a set of seamless texture variations randomly distributed over the surface at shading time. When compared to regular 2D textures, the inter-dimensional coherence of the 4D ABTF material model poses entirely new challenges to texture synthesis, which includes maintaining the consistency of material behavior throughout the 4D space spanned by the spatial image domain and the angular illumination hemisphere. In addition, we tackle the increased memory consumption caused by the numerous variations through a fitting scheme specifically designed to reconstruct the most prominent effects captured in the material model.

Image completion techniques are often used to fill in damaged images. In this study, a novel method is proposed that can handle large hole regions surrounded by different types of texture. This method is based on reconstructing damaged texture structure and texture synthesizing. The images are segmented into several regions and similar regions touching the hole region are linked, thereby resulting in new regions. It is found that the new regions provide sufficient information to generate a guidance map, which serves as a soft constraint in the inpainting process. With the guidance map, the hole is filled by using a global optimization texture-synthesis method. The experimental results illustrate that this novel method is a potential tool for image completion.

Large scenes such as building facades and other architectural constructions often contain repeating elements such as identical windows and brick patterns. In this paper, we present a novel approach that improves the resolution and geometry of 3D meshes of large scenes with such repeating elements. By leveraging structure from motion reconstruction and an off-the-shelf depth sensor, our approach captures a small sample of the scene in high resolution and automatically extends that information to similar regions of the scene. Using RGB and SfM depth information as a guide and simple geometric primitives as canvas, our approach extends the high-resolution mesh by exploiting powerful, image-based texture synthesis approaches. The final results improve on standard SfM reconstruction with higher detail. Our approach benefits from reduced manual labor as opposed to full RGBD reconstruction, and can be done much more cheaply than with LiDAR-based solutions.

In this work, we present a new framework for the stylization of text-based binary images. First, our method stylizes the stroke-based geometric shape like text, symbols and icons in the target binary image based on an input style image. Second, the composition of the stylized geometric shape and a background image is explored. To accomplish the task, we propose legibility-preserving structure and texture transfer algorithms, which progressively narrow the visual differences between the binary image and the style image. The stylization is then followed by a context-aware layout design algorithm, where cues for both seamlessness and aesthetics are employed to determine the optimal layout of the shape in the background. Given the layout, the binary image is seamlessly embedded into the background by texture synthesis under a context-aware boundary constraint. According to the contents of binary images, our method can be applied to many fields. We show that the proposed method is capable of addressing the unsupervised text stylization problem and is superior to state-of-the-art style transfer methods in automatic artistic typography creation. Besides, extensive experiments on various tasks, such as visual-textual presentation synthesis, icon/symbol rendering and structure-guided image inpainting, demonstrate the effectiveness of the proposed method., Comment: Accepted by IEEE Trans. on Image Processing. Project page: http://www.icst.pku.edu.cn/struct/Projects/UTS.html

We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video. For the large differences between pareidolia face reenactment and traditional human face reenactment, two main challenges are introduced, i.e., shape variance and texture variance. In this work, we propose a novel Parametric Unsupervised Reenactment Algorithm to tackle these two challenges. Specifically, we propose to decompose the reenactment into three catenate processes: shape modeling, motion transfer and texture synthesis. With the decomposition, we introduce three crucial components, i.e., Parametric Shape Modeling, Expansionary Motion Transfer and Unsupervised Texture Synthesizer, to overcome the problems brought by the remarkably variances on pareidolia faces. Extensive experiments show the superior performance of our method both qualitatively and quantitatively. Code, model and data are available on our project page1.

In the traditional cultural relic image digital repair system, there are obvious color aberrations and structural deviations between the restored part and the surrounding part, which weakens the visual connectivity of the restored result. In view of this, this research designs a digital repair system of damaged cultural relics image in professional training of restoration. In the hardware of the system, an integrated fast spherical camera is designed, and the transformation between image and pixel coordinate system is expressed by homogeneous coordinate system according to its parameters. In the system software, the damaged areas of cultural relics images are segmented and marked first, and the optimization algorithm of texture synthesis and restoration based on samples is designed. The experimental test results show that the system in this paper can ensure the use of the known information of the original image to the greatest extent, and make the repair result more reasonable on the basis of the minimum modification of the original image, so as to achieve the effect of professional practical training of repair.

Traditional visual secret sharing (VSS) strategies hide secret images in shares that are either imprinted on transparencies or are encoded and put away in a digital form. The shares can appear as noise-like pixels or as significant pictures; but it will arouse suspicion and increase interception risk during transmission of the shares. Subsequently, VSS schema experience from a transmission risk problem for the secret itself and for the members who are associated with the VSS conspire. To address this issue, proposed a advanced technique for advanced digital watermarking using a texture and also a natural-image-based VSS scheme (VSS scheme) that shares secret images via various carrier media to protect the secret and the participants during the transmission phase. Devise the texture synthesis process into digital image to hide secret messages. In comparison to using an existing cover image to hide messages, our algorithm hides the source texture image and embeds secret messages through the process of watermarking. The regular offers can be photographs or hand-painted pictures in computerized structure or in printed structure. We likewise propose potential approaches to conceal the key to diminish the transmission chance issue for the offer. Test results demonstrate that the proposed approach is an amazing answer for taking care of the transmission chance issue for the VSS technique.

International audience; Textures improve the appearance of virtual scenes by mapping visual details on the surface of 3D objects. Various scenarios – such as real-time animation, interactive texture modelling, or offline post-production – require textures to be deformed in a controllable and plausible manner. We propose a novel approach to model and control texture deformations, which is easy to implement in a standard graphics pipeline. The deformation is implemented at pixel resolution as a warping in the parametric domain. The warping is controlled locally and dynamically by real-time integration along the streamlines of a pre-computed flow field. We propose a technique to pre-compute the flow field from a simple scalar map representing heterogeneous dynamic behaviors. Moreover, to manage sampling issues arising in over-stretched areas during deformation, we provide a mechanism based on re-sampling and texture synthesis. Warping may alternatively be controlled by deformation of the underlying surface, environment parameters or interactive editing, which demonstrates the versatility of our approach.

The polynomial-based image secret sharing (ISS) scheme encodes a secret image into n shadows assigned to n participants. The secret image with high resolution is decoded by Lagrange interpolation when collecting any k or more shadows. Thus, ISS is used in applications such as distributive storage in the cloud, digital watermarking, block chain, and access control. Meaningful shadows are significant in ISS because meaningful shadows decrease the suspicion of image encryption and increase the efficiency of shadow management. Generally, previously meaningful ISS schemes were achieved through embedding the shadows into cover images using information hiding techniques and suffer from large pixel expansion and complex decoding procedure. Digital image processing, such as inpainting (texture synthesis), is a standard technique in multimedia applications. It will be highly significant if ISS can be performed in the processing of a normal digital image processing technique. Generally, the encoding method of an ISS scheme entails the use of a mathematical function that is sensitive to any slight change in the ISS output; therefore, the development of a method for performing the ISS procedure and simultaneously achieving image processing behavior is a key challenge. In this paper, we exploit the behavior ISS (BISS) and realize an image inpainting-based BISS scheme for the (k, n) threshold. Using screening operations, a secret image is encoded into the pixels of cover images by polynomial-based ISS in the processing of inpainting shadows to obtain meaningful shadows similar to the input cover images. In addition, the secret image can be losslessly decoded by Lagrange interpolation when collecting any k or more shadows. Experiments are given to confirm the efficiency of the scheme.

Military camouflage should be various in order to quickly adapt environmental changes of battlefield. The elusiveness is depressed if the constant camouflage patterns of the clothes and vehicles are used, as applying in the most of current military. In this paper, we propose a texture synthesis method from an image using convolutional neural networks, which has an effect on the camouflage generation on a 3D surface. We use the latest advances in style transitions in 2D images and map characteristic textures onto 3D surfaces according to geometric feature. This allows us to make an adaptive texture map of the 3D objects, even in the case of complex topologies.

Image inpainting was generally done physically by artists to remove deformity from works of art and photos. To cover the area of target or missing data from a signal, utilizing encompassing details and restructure signal is the fundamental job of image inpainting algorithms. We have considered and audited numerous distinct algorithms available for image inpainting and clarified their methodology. This paper includes various works in the branch of image inpainting and will help beginners who want to work and develop the image inpainting techniques.

Recently, deep generative adversarial networks for image generation have advanced rapidly; yet, only a small amount of research has focused on generative models for irregular structures, particularly meshes. Nonetheless, mesh generation and synthesis remains a fundamental topic in computer graphics. In this work, we propose a novel framework for synthesizing geometric textures. It learns geometric texture statistics from local neighborhoods (i.e., local triangular patches) of a single reference 3D model. It learns deep features on the faces of the input triangulation, which is used to subdivide and generate offsets across multiple scales, without parameterization of the reference or target mesh. Our network displaces mesh vertices in any direction (i.e., in the normal and tangential direction), enabling synthesis of geometric textures, which cannot be expressed by a simple 2D displacement map. Learning and synthesizing on local geometric patches enables a genus-oblivious framework, facilitating texture transfer between shapes of different genus., SIGGRAPH 2020

Deep learning method have been offering promising solutions for medical image processing, but failing to understand what features in the input image are captured and whether certain artifacts are mistakenly included in the model, thus create crucial problems in generalizability of the model. We targeted a common issue of this kind caused by manual annotations appeared in medical image. These annotations are usually made by the doctors at the spot of medical interest and have adversarial effect on many computer vision AI tasks. We developed an inpainting algorithm to remove the annotations and recover the original images. Besides we applied variational information bottleneck method in order to filter out the unwanted features and enhance the robustness of the model. Our impaiting algorithm is extensively tested in object detection in thyroid ultrasound image data. The mAP (mean average precision, with IoU = 0.3) is 27% without the annotation removal. The mAP is 83% if manually removed the annotations using Photoshop and is enhanced to 90% using our inpainting algorithm. Our work can be utilized in future development and evaluation of artificial intelligence models based on medical images with defects.

In this paper, a scheme of facial texture synthesis and 3D reconstruction based on 2D face image and 3D model under the Unity platform is proposed. First of all, this scheme uses the ASM algorithm to extract the key points of the human face. Next, the facial features in the face image and the standard skin color image are texture blended to get the texture of fused facial image. Finally, the texture of fused facial image is mapped to the 3D face model by texture mapping, and a better personalized 3D face reconstruction effect is obtained by this method. Compared with FaceGen, a software for modeling facial photos, the experimental results show that the proposed algorithm is superior to other existing algorithms in efficiency and has certain application value.

A back-of-the-napkin idea is typically created on the spur of the moment and captured via a few hand-sketched notes on whatever material is available, which often happens to be an actual paper napkin. This paper explores the preservation of such back-of-the-napkin ideas. Hand-sketched notes, reflecting those flashes of inspiration, are not limited to text; they can also include drawings and graphics. Napkin backgrounds typically exhibit diverse textural and colour motifs/patterns that may have high visual saliency from a low-level vision standpoint. We thus frame the extraction of hand-sketched notes as a background modelling and removal task. We propose a novel document background model based on texture mixtures constructed from the document itself via texture synthesis, which allows us to identify background pixels and extract hand-sketched data as foreground elements. Experiments on a novel napkin image dataset yield excellent results and showcase the robustness of our method with respect to the napkin contents. A texture-based background modelling approach, such as ours, is generic enough to cope with any type of hand-sketched notes.

With the rising interest in personalized VR and gaming experiences comes the need to create high quality 3D avatars that are both low-cost and variegated. Due to this, building dynamic avatars from a single unconstrained input image is becoming a popular application. While previous techniques that attempt this require multiple input images or rely on transferring dynamic facial appearance from a source actor, we are able to do so using only one 2D input image without any form of transfer from a source image. We achieve this using a new conditional Generative Adversarial Network design that allows fine-scale manipulation of any facial input image into a new expression while preserving its identity. Our photoreal avatar GAN (paGAN) can also synthesize the unseen mouth interior and control the eye-gaze direction of the output, as well as produce the final image from a novel viewpoint. The method is even capable of generating fully-controllable temporally stable video sequences, despite not using temporal information during training. After training, we can use our network to produce dynamic image-based avatars that are controllable on mobile devices in real time. To do this, we compute a fixed set of output images that correspond to key blendshapes, from which we extract textures in UV space. Using a subject's expression blendshapes at run-time, we can linearly blend these key textures together to achieve the desired appearance. Furthermore, we can use the mouth interior and eye textures produced by our network to synthesize on-the-fly avatar animations for those regions. Our work produces state-of-the-art quality image and video synthesis, and is the first to our knowledge that is able to generate a dynamically textured avatar with a mouth interior, all from a single image.

Image inpainting is a process of reconstructing missing regions, or removing unwanted objects automatically by propagating intensity and texture information from surrounding parts of the image in a visually plausible manner. We propose a new exemplar-based image inpainting algorithm, which uses the recently developed metric called the perceptual-fidelity aware mean squared error (PAMSE). The PAMSE is a Gaussian-smoothed mean squared error (MSE) and approximates a weighted sum of the gradient of MSE, the Laplacian of MSE, and MSE itself. We show that, compared to MSE, PAMSE is a promising perceptual fidelity metric for application to image inpainting and leads to better performance in propagating texture and geometric structure simultaneously.

Natural textures with gradually changed illumination bring great troubles to the field of image inpainting. Using the improved dynamical modeling approach, an energy function model gradually changed from the center to the boundary of data missing region of image is established. On this basis, a gradually changed directional priority function is proposed to ensure the gradual propagation of texture synthesis. In addition, to achieve the feasible propagation of the boundary between different textural regions, the relative test point selection, as well as energy function modeling approach, is also discussed for different cases. Besides, the established energy function models are theoretically evaluated to demonstrate their accuracy in images. Experimental results show the effectiveness of the proposed approach in the natural images with gradually changed illumination.

We introduce a method to stylize photographs with auxiliary textures, by means of the Laplacian pyramid. Laplacian pyramid coefficients from a synthetic texture are combined with the coefficients from the original image by means of a smooth maximum function. The final result is a stylized image which maintains the structural characteristics from the input, including edges, color, and existing texture, while enhancing the image with additional fine-scale details. Further, we extend patch-based texture synthesis to include a guidance channel so that texture structures are aligned with an orientation field, obtained through the image structure tensor.

Iris recognition in the presence of eyelash occlusions is a challenging task over the years since it has begun. The active area captured under non-ideal imaging conditions usually suffers from low contrast, poor brightness, blur due to camera or subject’s relative motion and particularly eyelash and eyelid occlusions. Accurate segmentation methods avoid occlusions to some extent but not completely always. In the proposed method, pixel-wise texture synthesis is done on occluded regions which improves the correct recognition rate (CRR). The contourlet transform which is a multiresolution tool, decomposes an image into different scales and directions with the help of pyramidal and directional filter bank (PDFB). A new FIR filter named as SSK filter is proposed by the authors for the PDFB in contourlets to extract apt features of iris such that the CRR is further improved. The performance of the proposed method is checked against CASIA-Iris-Interval (V3), IITD, CASIA-V1 and UBIRIS-V1 iris databases, and from the results obtained, it is proved that the proposed method is very much worthwhile for improved iris recognition even in the presence of eyelash and eyelid occlusions.

Endowing 3D objects with realistic surface appearance is a challenging and time-demanding task, as real-world surfaces typically exhibit a plethora of spatially variant geometric and photometric detail. Not surprisingly, computer artists commonly use images of real-world objects as an inspiration and a reference for their digital creations. However, despite two decades of research on image-based modeling, there are still no tools available for automatically extracting the detailed appearance (microgeometry and texture) of a 3D surface from a single image. In this article, we present a novel user-assisted approach for quickly and easily extracting a nonparametric appearance model from a single photograph of a reference object. The extraction process requires a user-provided proxy, whose geometry roughly approximates that of the object in the image. Since the proxy is just a rough approximation, it is necessary to align and deform it so as to match the reference object. The main contribution of this work is a novel technique to perform such an alignment, which enables accurate joint recovery of geometric detail and reflectance. The correlations between the recovered geometry at various scales and the spatially varying reflectance constitute a nonparametric appearance model. Once extracted, the appearance model may then be applied to various 3D shapes, whose large-scale geometry may differ considerably from that of the original reference object. Thus, our approach makes it possible to construct an appearance library, allowing users to easily enrich detail-less 3D shapes with realistic geometric detail and surface texture.

Shamir’s polynomial-based secret image sharing (SIS) scheme and visual secret sharing (VSS) also called visual cryptography scheme (VCS), are the primary branches in SIS. In traditional (k, n) threshold secret sharing, a secret image is fully (entirely) generated into n shadow images (shares) distributed to n associated participants. The secret image can be recovered by collecting any k or more shadow images. The previous SIS schemes dealt with the full secret image neglecting the possible situation that only part of the secret image needs protection. However, in some applications, only target part of the secret image may need to be protected while other parts may be not in a full image. In this paper, we consider the partial secret image sharing (PSIS) issue as well as propose a PSIS scheme for (n, n) threshold based on image inpainting and linear congruence (LC). First the target part is manually selected or marked in the color secret image. Second, the target part is automatically removed from the original secret image to obtain the same input cover images (unpainted shadow images). Third, the target secret part is generated into the pixels corresponding to shadow images by LC in the processing of shadow images texture synthesis (inpainting), so as to obtain the shadow images in a visually plausible way. As a result, the full secret image including the target secret part and other parts will be recovered losslessly by adding all the inpainted meaningful shadow images. Experiments are conducted to evaluate the efficiency of the proposed scheme.

Transferring the style of an example image to a content image opens the door of artistic creation for end users. However, it is especially challenging for portrait photos since human vision system is sensitive to the slight artifacts on portraits. Previous methods use facial landmarks to densely align the content face with the style face to reduce the artifacts. However, they can only handle the facial region. As for the whole image, building the dense correspondence is difficult and may easily introduce errors. In this paper, we propose a robust approach for portrait style transfer that gets rid of dense correspondence. Our approach is based on the guided image synthesis framework. We propose three novel guidance maps for the synthesis process. Contrary to former methods, these maps do not require the dense correspondence between content image and style image, which allows our method to handle the whole portrait photo instead of facial region only. In comparison with recent neural style transfer methods, our method achieves more pleasing results and preserves more texture details. Extensive experiments demonstrate our advantage over former methods on portrait style transfer.

We present a fully automatic framework that digitizes a complete 3D head with hair from a single unconstrained image. Our system offers a practical and consumer-friendly end-to-end solution for avatar personalization in gaming and social VR applications. The reconstructed models include secondary components (eyes, teeth, tongue, and gums) and provide animation-friendly blendshapes and joint-based rigs. While the generated face is a high-quality textured mesh, we propose a versatile and efficient polygonal strips (polystrips) representation for the hair. Polystrips are suitable for an extremely wide range of hairstyles and textures and are compatible with existing game engines for real-time rendering. In addition to integrating state-of-the-art advances in facial shape modeling and appearance inference, we propose a novel single-view hair generation pipeline, based on 3D-model and texture retrieval, shape refinement, and polystrip patching optimization. The performance of our hairstyle retrieval is enhanced using a deep convolutional neural network for semantic hair attribute classification. Our generated models are visually comparable to state-of-the-art game characters designed by professional artists. For real-time settings, we demonstrate the flexibility of polystrips in handling hairstyle variations, as opposed to conventional strand-based representations. We further show the effectiveness of our approach on a large number of images taken in the wild, and how compelling avatars can be easily created by anyone.

Near-regular texture is a common feature for both nature scene and 3D human models. However, traditional texture synthesis has only been able to produce a single result. This study proposes a parallel, and interactive, method for the texture synthesis of irregular appearance in 3D human models. The new method includes two major procedures: (1) iterative edge extraction and processing, and (2) parallel texture synthesis, which generates a texture with higher quality. In this paper, the effectiveness of near-regular texture synthesis algorithm is demonstrated experimentally.

We introduce a novel approach to example-based stylization of portrait videos that preserves both the subject's identity and the visual richness of the input style exemplar. Unlike the current state-of-the-art based on neural style transfer [Selim et al. 2016], our method performs non-parametric texture synthesis that retains more of the local textural details of the artistic exemplar and does not suffer from image warping artifacts caused by aligning the style exemplar with the target face. Our method allows the creation of videos with less than full temporal coherence [Ruder et al. 2016]. By introducing a controllable amount of temporal dynamics, it more closely approximates the appearance of real hand-painted animation in which every frame was created independently. We demonstrate the practical utility of the proposed solution on a variety of style exemplars and target videos.

The realistic synthesis and rendering of film grain is a crucial goal for many amateur and professional photographers and film-makers whose artistic works require the authentic feel of analog photography. The objective of this work is to propose an algorithm that reproduces the visual aspect of film grain texture on any digital image. Previous approaches to this problem either propose unrealistic models or simply blend scanned images of film grain with the digital image, in which case the result is inevitably limited by the quality and resolution of the initial scan. In this work, we introduce a stochastic model to approximate the physical reality of film grain, and propose a resolution-free rendering algorithm to simulate realistic film grain for any digital input image. By varying the parameters of this model, we can achieve a wide range of grain types. We demonstrate this by comparing our results with film grain examples from dedicated software, and show that our rendering results closely resemble these real film emulsions. In addition to realistic grain rendering, our resolution-free algorithm allows for any desired zoom factor, even down to the scale of the microscopic grains themselves.

In this paper, a new algorithm is presented for the nonparametric synthesis of arbitrary-shaped textures from an initial texture sample, called the exemplar, and a reference orientation map, called the reference and intended to constrain the coarse scale orientation flow of the output texture. The synthesis process consists of three stages. First, a dictionary is constructed, composed of rotated versions of the exemplar and of their structure tensor fields. Then a synthetic structure tensor field is constructed from the exemplar’s one, under the constraint of the reference. Finally, the output texture is synthesized from the previously generated structure tensor field, again under the constraint of the reference. Synthesis experiments show that the proposed method is able to faithfully reproduce the visual aspect of input samples while respecting the large-scale orientation flow of the reference. This paves the way toward the synthesis of 3-D textures of arbitrary shapes from 2-D exemplars, with applications in virtual material design.