Search

Object detection is widely used in object tracking; anchor-free object tracking provides an end-to-end single-object-tracking approach. In this study, we propose a new anchor-free network, the Siamese center-prediction network (SiamCPN). Given the presence of referenced object features in the initial frame, we directly predict the center point and size of the object in subsequent frames in a Siamese-structure network without the need for perframe post-processing operations. Unlike other anchor-free tracking approaches that are based on semantic segmentation and achieve anchor-free tracking by pixel-level prediction, SiamCPN directly obtains all information required for tracking, greatly simplifying the model. A center-prediction sub-network is applied to multiple stages of the backbone to adaptively learn from the experience of different branches of the Siamese net. The model can accurately predict object location, implement appropriate corrections, and regress the size of the target bounding box. Compared to other leading Siamese networks, SiamCPN is simpler, faster, and more efficient as it uses fewer hyperparameters. Experiments demonstrate that our method outperforms other leading Siamese networks on GOT-10K and UAV123 benchmarks, and is comparable to other excellent trackers on LaSOT, VOT2016, and OTB-100 while improving inference speed 1.5 to 2 times.

Distinguishing aesthetically pleasing food photos from others is an important visual analysis task for social media and ranking systems related to food. Nevertheless, aesthetic assessment of food images remains a challenging and relatively unexplored task, largely due to the lack of related food image datasets and practical knowledge. Thus, we present the Gourmet Photography Dataset (GPD), the first large-scale dataset for aesthetic assessment of food photos. It contains 24,000 images with corresponding binary aesthetic labels, covering a large variety of foods and scenes. We also provide a non-stationary regularization method to combat over-fitting and enhance the ability of tuned models to generalize. Quantitative results from extensive experiments, including a generalization ability test, verify that neural networks trained on the GPD achieve comparable performance to human experts on the task of aesthetic assessment. We reveal several valuable findings to support further research and applications related to visual aesthetic analysis of food images. To encourage further research, we have made the GPD publicly available at https://github.com/Openning07/GPA.

Input scale plays an important role in modern detection frameworks, and an optimal training scale for images exists empirically. However, the optimal one usually cannot be reached in facing extremely large images under the memory constraint. In this study, we explore the scale effect inside the object detection pipeline and find that feature upsampling with the introduction of high-resolution information benefits the detection. Compared with direct input upscaling, feature upsampling trades a small performance loss for a large amount of memory savings. From these observations, we propose a self-supervised feature augmentation network, which takes downsampled images as inputs and aims to generate comparable features with the ones when feeding upscaled images to networks. We present a guided feature upsampling module, which takes downsampled images as inputs, to learn upscaled feature representations with the supervision of real large features acquired from upscaled images. In a self-supervised learning manner, we can introduce detailed information of images to the network. For an efficient feature upsampling, we design a residualized sub-pixel convolution block based on a sub-pixel convolution layer, which involves considerable information in upsampling process. Experiments on Mapillary Vistas Dataset (MVD), Cityscapes, and COCO are conducted to demonstrate the effectiveness of our method. On the MVD and Cityscapes detection benchmarks, in which the images are extremely large, our method surpasses current approaches. On COCO, the proposed method obtains comparable results to existing methods but with higher efficiency.

ID face photos are widely used for identity verification in many business authentication situations. To avoid any infringement and misuse, the ID photos provided by the relevant government agencies and business organizations are always corrupted with designed watermarks, such as random wave lines or meshes. These corrupted images are further compressed with JPEG algorithm to reduce their storage size. The artifacts caused by the random meshes and JPEG compression seriously destroy the original image information and quality, which makes the face verification between the corrupted ID faces and daily life images extremely difficult. To tackle these issues, a preprocessing step called blind inpainting is needed to recover the corrupted ID faces. In this paper, we present a new framework to address this blind face inpainting problem. We use an improved Deep Recursive Residual Network (IDRRN) to learn an effective non-linear mapping between the corrupted and clean ID image pairs. To train the IDRRN model, a unified Euclidean loss function considering both 0- and 1st-order pixel residuals is proposed to enhance the image pixel as well as gradient reconstruction. In addition, we collect a dataset of clean ID images and develop a simulation procedure to generate corresponding corrupted ID face images. Final experiments demonstrate that the recovered ID face images inferred from our IDRRN model achieve the best performance in terms of perceptual error and verification accuracy.

Facial action units (AUs) are defined to depict movements of facial muscles, which are basic elements to encode facial expressions. Automatic AU intensity estimation is an important task in affective computing. Previous works leverage the representation power of deep neural networks (DNNs) to improve the performance of intensity estimation. However, a large number of intensity annotations are required to train DNNs that contain millions of parameters. But it is expensive and difficult to build a large-scale database with AU intensity annotation since AU annotation requires annotators have strong domain expertise. We propose a novel semi-supervised deep convolutional network that leverages extremely limited AU annotations for AU intensity estimation. It requires only intensity annotations of keyframes of training sequences. Domain knowledge on AUs is leveraged to provide weak supervisory information, including relative appearance similarity, temporal intensity ordering, facial symmetry, and contrastive appearance difference. We also propose a strategy to train a model for joint intensity estimation of multiple AUs under the setting of semi-supervised learning, which greatly improves the efficiency during inference. We perform empirical experiments on two public benchmark expression databases and make comparisons with state-of-the-art methods to demonstrate the effectiveness of the proposed method.

Accurate building rooftop extraction from high-resolution aerial images is of crucial importance in a wide range of applications. Owing to the varying appearance and large-scale range of scene objects, especially for building rooftops in different scales and heights, single-scale or individual prior-based extraction technique is insufficient in pursuing efficient, generic, and accurate extraction results. The trend toward integrating multiscale or several cue techniques appears to be the best way; thus, such integration is the focus of this paper. We first propose a novel salient rooftop detector integrating four correlative RGB-D priors (depth cue, uniqueness prior, shape prior, and transition surface prior) for improved rooftop extraction to address the preceding complex issues mentioned. Then, these correlative cues are computed from image layers created by our multilevel segmentation and further fused into the state-of-the-art high-order conditional random field (CRF) framework to locate the rooftop. Finally, an iterative optimization strategy is applied for high-quality solving, which can robustly handle varying appearance of building rooftops. Performance evaluations in the SZTAKI-INRIA benchmark data sets show that our method outperforms the traditional color-based algorithm and the original high-order CRF algorithm and its variants. The proposed algorithm is also evaluated and found to produce consistently satisfactory results for various large-scale, real-world data sets.

The ability to learn more concepts from incrementally arriving data over time is essential for the development of a lifelong learning system. However, deep neural networks often suffer from forgetting previously learned concepts when continually learning new concepts, which is known as the catastrophic forgetting problem. The main reason for catastrophic forgetting is that past concept data are not available, and neural weights are changed during incrementally learning new concepts. In this article, we propose an incremental concept learning framework that includes two components, namely, ICLNet and RecallNet. ICLNet, which consists of a trainable feature extractor and a dynamic concept memory matrix, aims to learn new concepts incrementally. We propose a concept-contrastive loss to alleviate the magnitude of neural weight changes and mitigate the catastrophic forgetting problems. RecallNet aims to consolidate old concepts memory and recall pseudo samples, whereas ICLNet learns new concepts. We propose a balanced online memory recall strategy to reduce the information loss of old concept memory. We evaluate the proposed approach on the MNIST, Fashion-MNIST, and SVHN data sets and compare it with other pseudorehearsal-based approaches. Extensive experiments demonstrate the effectiveness of our approach.

Object detection has achieved remarkable progress in the past decade. However, the detection of oriented and densely packed objects remains challenging because of following inherent reasons: (1) receptive fields of neurons are all axis-aligned and of the same shape, whereas objects are usually of diverse shapes and align along various directions; (2) detection models are typically trained with generic knowledge and may not generalize well to handle specific objects at test time; (3) the limited dataset hinders the development on this task. To resolve the first two issues, we present a dynamic refinement network that consists of two novel components, i.e., a feature selection module (FSM) and a dynamic refinement head (DRH). Our FSM enables neurons to adjust receptive fields in accordance with the shapes and orientations of target objects, whereas the DRH empowers our model to refine the prediction dynamically in an object-aware manner. To address the limited availability of related benchmarks, we collect an extensive and fully annotated dataset, namely, SKU110K-R, which is relabeled with oriented bounding boxes based on SKU110K. We perform quantitative evaluations on several publicly available benchmarks including DOTA, HRSC2016, SKU110K, and our own SKU110K-R dataset. Experimental results show that our method achieves consistent and substantial gains compared with baseline approaches. The code and dataset are available at https://github.com/Anymake/DRN_CVPR2020., Comment: Accepted by CVPR 2020 as Oral

Multimodal and multi-domain stylization are two important problems in the field of image style transfer. Currently, there are few methods that can perform both multimodal and multi-domain stylization simultaneously. In this paper, we propose a unified framework for multimodal and multi-domain style transfer with the support of both exemplar-based reference and randomly sampled guidance. The key component of our method is a novel style distribution alignment module that eliminates the explicit distribution gaps between various style domains and reduces the risk of mode collapse. The multimodal diversity is ensured by either guidance from multiple images or random style code, while the multi-domain controllability is directly achieved by using a domain label. We validate our proposed framework on painting style transfer with a variety of different artistic styles and genres. Qualitative and quantitative comparisons with state-of-the-art methods demonstrate that our method can generate high-quality results of multi-domain styles and multimodal instances with reference style guidance or random sampled style.

Image enhancement is an important branch in the field of image processing. A few existing methods leverage Generative Adversarial Networks (GANs) for this task. However, they have several defects when applied to a specific type of images, such as food photo. First, a large set of original-enhanced image pairs are required to train GANs that have millions of parameters. Such image pairs are expensive to acquire. Second, color distribution of enhanced images generated by previous methods is not consistent with the original ones, which is not expected. To alleviate the issues above, we propose a novel method for food photo enhancement. No original-enhanced image pairs are required except only original images. We investigate Food Faithful Color Semantic Rules in Enhanced Dataset Photo Enhancement (Faith-EDPE) and also carefully design a light generator which can preserve semantic relations among colors. We evaluate the proposed method on public benchmark databases to demonstrate the effectiveness of the proposed method through visual results and user studies.

This study introduces a novel conditional recycle generative adversarial network for facial attribute transformation, which can transform high-level semantic face attributes without changing the identity. In our approach, we input a source facial image to the conditional generator with target attribute condition to generate a face with the target attribute. Then we recycle the generated face back to the same conditional generator with source attribute condition. A face which should be similar to that of the source face in personal identity and facial attributes is generated. Hence, we introduce a recycle reconstruction loss to enforce the final generated facial image and the source facial image to be identical. Evaluations on the CelebA dataset demonstrate the effectiveness of our approach. Qualitative results show that our approach can learn and generate high-quality identity-preserving facial images with specified attributes.

Image retargeting as a basic trick has been successfully applied to various computer vision problems. In this work, we propose a fast non-continuous seam carving with seam context (FN2SC) for robust and efficient image retargeting. FN2SC first conducts content aware image partition to separate an image into several strips with different target sizes from their content importance. It helps the removed seams in key objects distribute in a relatively uniform manner and prevents distortions to key objects. Then FN2SC performs fast non-continuous seam carving controlled by seam context of both neighboring relationship and touch bound relationship of seams. The seam context makes the removed seams distribute scattered and relieves artifacts caused by seam carving. During seam carving, image distortion is monitored by fast content aware image distance. Finally, FN2SC switches seam carving to scaling when the distortion meets the tolerance, which resizes the strips to the target sizes for image retargeting. Specifically, fast seam searching and image distortion based switching make FN2SC a fast and effective hybrid scheme. Experimental results demonstrate that the proposed FN2SC approach achieves good performance in terms of image quality and efficiency comprehensively.

Similar objects are ubiquitous and abundant in both natural and artificial scenes. Determining the visual importance of several similar objects in a complex photograph is a challenge for image understanding algorithms. This study aims to define the importance of similar objects in an image and to develop a method that can select the most important instances for an input image from multiple similar objects. This task is challenging because multiple objects must be compared without adequate semantic information. This challenge is addressed by building an image database and designing an interactive system to measure object importance from human observers. This ground truth is used to define a range of features related to the visual importance of similar objects. Then, these features are used in learning-to-rank and random forest to rank similar objects in an image. Importance predictions were validated on 5,922 objects. The most important objects can be identified automatically. The factors related to composition (e.g., size, location, and overlap) are particularly informative, although clarity and color contrast are also important. We demonstrate the usefulness of similar object importance on various applications, including image retargeting, image compression, image re-attentionizing, image admixture, and manipulation of blindness images.

Facial action unit (AU) intensity is an index to characterize human expressions. Accurate AU intensity estimation depends on three major elements: image representation, intensity estimator, and supervisory information. Most existing methods learn intensity estimator with fixed image representation, and rely on the availability of fully annotated supervisory information. In this paper, a novel general framework for AU intensity estimation is presented, which differs from traditional estimation methods in two aspects. First, rather than keeping image representation fixed, it simultaneously learns representation and intensity estimator to achieve an optimal solution. Second, it allows incorporating weak supervisory training signal from human knowledge (e.g. feature smoothness, label smoothness, label ranking, and positive label), which makes our model trainable even fully annotated information is not available. More specifically, human knowledge is represented as either soft or hard constraints which are encoded as regularization terms or equality/inequality constraints, respectively. On top of our novel framework, we additionally propose an efficient algorithm for optimization based on Alternating Direction Method of Multipliers (ADMM). Evaluations on two benchmark databases show that our method outperforms competing methods under different ratios of AU intensity annotations, especially for small ratios.

Visual aesthetic assessment has been an active research field for decades. Although latest methods have achieved promising performance on benchmark datasets, they typically rely on a large number of manual annotations including both aesthetic labels and related image attributes. In this paper, we revisit the problem of image aesthetic assessment from the self-supervised feature learning perspective. Our motivation is that a suitable feature representation for image aesthetic assessment should be able to distinguish different expert-designed image manipulations, which have close relationships with negative aesthetic effects. To this end, we design two novel pretext tasks to identify the types and parameters of editing operations applied to synthetic instances. The features from our pretext tasks are then adapted for a one-layer linear classifier to evaluate the performance in terms of binary aesthetic classification. We conduct extensive quantitative experiments on three benchmark datasets and demonstrate that our approach can faithfully extract aesthetics-aware features and outperform alternative pretext schemes. Moreover, we achieve comparable results to state-of-the-art supervised methods that use 10 million labels from ImageNet., Comment: AAAI Conference on Artificial Intelligence, 2020, accepted

In this study, we present the Gourmet Photography Dataset (GPD), which is the first large-scale dataset for aesthetic assessment of food photographs. We collect 12,000 food images together with human-annotated labels (i.e., aesthetically positive or negative) to build this dataset. We evaluate the performance of several popular machine learning algorithms for aesthetic assessment of food images to verify the effectiveness and importance of our GPD dataset. Experimental results show that deep convolutional neural networks trained on GPD can achieve comparable performance with human experts in this task, even on unseen food photographs. Our experiments also provide insights to support further study and applications related to visual analysis of food images.

In this paper, we address the problem of constraint detection for layout regularization. The layout we consider is a set of two-dimensional elements where each element is represented by its bounding box. Layout regularization is important in digitizing plans or images, such as floor plans and facade images, and in the improvement of user-created contents, such as architectural drawings and slide layouts. To regularize a layout, we aim to improve the input by detecting and subsequently enforcing alignment, size, and distance constraints between layout elements. Similar to previous work, we formulate layout regularization as a quadratic programming problem. In addition, we propose a novel optimization algorithm that automatically detects constraints. We evaluate the proposed framework using a variety of input layouts from different applications. Our results demonstrate that our method has superior performance to the state of the art.

We present an automatic approach for symmetrizing urban facade layouts. Our method can generate a symmetric layout through minimally modifying the original input layout. Based on the principles of symmetry in urban design, we formulate facade layout symmetrization as an optimization problem. Our method further enhances the regularity of the final layout by redistributing and aligning elements in the layout. We demonstrate that the proposed solution can effectively generate symmetric facade layouts.

Squared forms of photos are widely used in social media as album covers or thumbnails of image streams. In this study, we realize photo squarization by modeling Retargeting Visual Perception Issues, which reflect human perception preference toward image ratargeting. General image retargeting techniques deal with three common issues, namely, salient content, object shape, and scene composition, to preserve the important information of original image. We propose a new way based on multi-operator techniques to investigate human behavior in balancing the three issues. We establish a new dataset and observe human behavior by inviting investigators to retarget images to square manually. We propose a data-driven approach composed of perception and distillation modules by using deep learning techniques to predict human perception preference. The perception part learns the relations among the three issues, and the distillation part transfers the learned relations to a simple but effective network. Our study contributes to deep learning literature by optimizing a network index and lightening its running burden. Experimental results show that photo squarization results generated by the proposed model are consistent with human visual perception results.

Aggregation structures with explicit information, such as image attributes and scene semantics, are effective and popular for intelligent systems for assessing aesthetics of visual data. However, useful information may not be available due to the high cost of manual annotation and expert design. In this paper, we present a novel multi-patch (MP) aggregation method for image aesthetic assessment. Different from state-of-the-art methods, which augment an MP aggregation network with various visual attributes, we train the model in an end-to-end manner with aesthetic labels only (i.e., aesthetically positive or negative). We achieve the goal by resorting to an attention-based mechanism that adaptively adjusts the weight of each patch during the training process to improve learning efficiency. In addition, we propose a set of objectives with three typical attention mechanisms (i.e., average, minimum, and adaptive) and evaluate their effectiveness on the Aesthetic Visual Analysis (AVA) benchmark. Numerical results show that our approach outperforms existing methods by a large margin. We further verify the effectiveness of the proposed attention-based objectives via ablation studies and shed light on the design of aesthetic assessment systems.

Facial action unit (AU) intensity estimation plays an important role in affective computing and human-computer interaction. Recent works have introduced deep neural networks for AU intensity estimation, but they require a large amount of intensity annotations. AU annotation needs strong domain expertise and it is expensive to construct a large database to learn deep models. We propose a novel knowledge-based semi-supervised deep convolutional neural network for AU intensity estimation with extremely limited AU annotations. Only the intensity annotations of peak and valley frames in training sequences are needed. To provide additional supervision for model learning, we exploit naturally existing constraints on AUs, including relative appearance similarity, temporal intensity ordering, facial symmetry, and contrastive appearance difference. Experimental evaluations are performed on two public benchmark databases. With around 2% of intensity annotations in FERA 2015 and around 1% in DISFA for training, our method can achieve comparable or even better performance than the state-of-the-art methods which use 100% of intensity annotations in the training set.

Automatic intensity estimation of facial action units (AUs) is challenging in two aspects. First, capturing subtle changes of facial appearance is quite difficult. Second, the annotation of AU intensity is scarce and expensive. Intensity annotation requires strong domain knowledge thus only experts are qualified. The majority of methods directly apply supervised learning techniques to AU intensity estimation while few methods exploit unlabeled samples to improve the performance. In this paper, we propose a novel weakly supervised regression model-Bilateral Ordinal Relevance Multi-instance Regression (BORMIR), which learns a frame-level intensity estimator with weakly labeled sequences. From a new perspective, we introduce relevance to model sequential data and consider two bag labels for each bag. The AU intensity estimation is formulated as a joint regressor and relevance learning problem. Temporal dynamics of both relevance and AU intensity are leveraged to build connections among labeled and unlabeled image frames to provide weak supervision. We also develop an efficient algorithm for optimization based on the alternating minimization framework. Evaluations on three expression databases demonstrate the effectiveness of the proposed method.

Facial action units (AUs) play an important role in human emotion understanding. One big challenge for data-driven AU recognition approaches is the lack of enough AU annotations, since AU annotation requires strong domain expertise. To alleviate this issue, we propose a knowledge-driven method for jointly learning multiple AU classifiers without any AU annotation by leveraging prior probabilities on AUs, including expression-independent and expression-dependent AU probabilities. These prior probabilities are drawn from facial anatomy and emotion studies, and are independent of datasets. We incorporate the prior probabilities on AUs as the constraints into the objective function of multiple AU classifiers, and develop an efficient learning algorithm to solve the formulated problem. Experimental results on five benchmark expression databases demonstrate the effectiveness of the proposed method, especially its generalization ability, and the power of the prior probabilities.

Real-world applications could benefit from the ability to automatically retarget an image to different aspect ratios and resolutions, while preserving its visually and semantically important content. However, not all images can be equally well processed that way. In this work, we introduce the notion of image retargetability to describe how well a particular image can be handled by content-aware image retargeting. We propose to learn a deep convolutional neural network to rank photo retargetability in which the relative ranking of photo retargetability is directly modeled in the loss function. Our model incorporates joint learning of meaningful photographic attributes and image content information which can help regularize the complicated retargetability rating problem. To train and analyze this model, we have collected a database which contains retargetability scores and meaningful image attributes assigned by six expert raters. Experiments demonstrate that our unified model can generate retargetability rankings that are highly consistent with human labels. To further validate our model, we show applications of image retargetability in retargeting method selection, retargeting method assessment and photo collage generation., Comment: 11 pages, 13 figures

In this paper, we present a method for reconstructing the drawing process of Chinese brush paintings. We demonstrate the possibility of computing an artistically reasonable drawing order from a static brush painting that is consistent with the rules of art. We map the key principles of drawing composition to our computational framework, which first organizes the strokes in three stages and then optimizes stroke ordering with natural evolution strategies. Our system produces reasonable animated constructions of Chinese brush paintings with minimal or no user intervention. We test our algorithm on a range of input paintings with varying degrees of complexity and structure and then evaluate the results via a user study. We discuss the applications of the proposed system to painting instruction, painting animation, and image stylization, especially in the context of art teaching. published

This article presents a framework for natural texture synthesis and processing. This framework is motivated by the observation that given examples captured in natural scene, texture synthesis addresses a critical problem, namely, that synthesis quality can be affected adversely if the texture elements in an example display spatially varied patterns, such as perspective distortion, the composition of different sub-textures, and variations in global color pattern as a result of complex illumination. This issue is common in natural textures and is a fundamental challenge for previously developed methods. Thus, we address it from a feature point of view and propose a feature-aware approach to synthesize natural textures. The synthesis process is guided by a feature map that represents the visual characteristics of the input texture. Moreover, we present a novel adaptive initialization algorithm that can effectively avoid the repeat and verbatim copying artifacts. Our approach improves texture synthesis in many images that cannot be handled effectively with traditional technologies.

International audience; We propose a new mechanism to pave the way for efficient learning against class-imbalance and improve representation of identity vector (i-vector) in automatic speaker verification (ASV). The insight is to effectively exploit the inherent structure within ASV corpus — centroid priori. In particular: (1) to ensure learning efficiency against class-imbalance, the centroid-aware balanced boosting sampling is proposed to collect balanced mini-batch; (2) to strengthen local discriminative modeling on the mini-batches, neighborhood component analysis (NCA) and magnet loss (MNL) are adopted in ASV-specific modifications. The integration creates adaptive NCA (AdaNCA) and linear MNL (LMNL). Numerical results show that LMNL is a competitive candidate for low-dimensional projection on i-vector (EER = 3.84% on SRE2008, EER = 1.81% on SRE2010), enjoying competitive edge over linear discriminant analysis (LDA). AdaNCA (EER = 4.03% on SRE2008, EER = 2.05% on SRE2010) also performs well. Furthermore, to facilitate the future study on boosting sampling, connections between boosting sampling, hinge loss and data augmentation have been established, which help understand the behavior of boosting sampling further.

This paper presents a data-driven approach for automatically generating cartoon faces in different styles from a given portrait image. Our stylization pipeline consists of two steps: an offline analysis step to learn about how to select and compose facial components from the databases; a runtime synthesis step to generate the cartoon face by assembling parts from a database of stylized facial components. We propose an optimization framework that, for a given artistic style, simultaneously considers the desired image-cartoon relationships of the facial components and a proper adjustment of the image composition. We measure the similarity between facial components of the input image and our cartoon database via image feature matching, and introduce a probabilistic framework for modeling the relationships between cartoon facial components. We incorporate prior knowledge about image-cartoon relationships and the optimal composition of facial components extracted from a set of cartoon faces to maintain a natural, consistent, and attractive look of the results. We demonstrate generality and robustness of our approach by applying it to a variety of portrait images and compare our output with stylized results created by artists via a comprehensive user study published

Chinese Calligraphy is a form of calligraphy widely practised in China and revered in the Chinese cultural sphere, including countries such as Japan, Korea, Vietnam, etc.. The variation of characters in shape, size and shade is the key skill in creation of Chinese calligraphic artworks. Characters should be carefully designed and composed together to create an excellent artwork. Previous works usually focus on synthesizing new calligraphic characters of a specific style [Xia et al. 2013], reproducing the writing processes of Chinese calligraphic artworks [Yang et al. 2014], or animating the construction process of Chinese ink-wash paintings [Tang et al. 2014], instead of creating and optimizing new calligraphic artworks. In this work, we develop a data-driven framework to optimally compose multiple characters picked from a database to form a Chinese Calligraphic artwork (see Figure 1). Our system can automatically generate a calligraphy in the style of a specific calligrapher. Currently we focus on the adjustment of character sizes in an artwork and formulate the composing of characters as a Markov process. The understanding of art varies with people, so our objective is finding a reasonable solution that is plausible to an artist's creating style, instead of searching for the best composing result. To the best of our knowledge, our work is the first attempt to optimize the creating process of Chinese calligraphic artworks.

Real-world images usually contain vivid contents and rich textural details, which will complicate the manipulation on them. In this paper, we design a new framework based on exampled-based texture synthesis to enhance content-aware image retargeting. By detecting the textural regions in an image, the textural image content can be synthesized rather than simply distorted or cropped. This method enables the manipulation of textural & non-textural regions with different strategies since they have different natures. We propose to retarget the textural regions by example-based synthesis and non-textural regions by fast multi-operator. To achieve practical retargeting applications for general images, we develop an automatic and fast texture detection method that can detect multiple disjoint textural regions. We adjust the saliency of the image according to the features of the textural regions. To validate the proposed method, comparisons with state-of-the-art image retargeting techniques and a user study were conducted. Convincing visual results are shown to demonstrate the effectiveness of the proposed method.

International audience; This paper presents an image-based algorithm for simulating the visual adaptation of the human visual system to various illuminations, especially in dark nighttime conditions. The human visual system exhibits different characteristics depending on the illumination intensity, with photopic vision in bright conditions, scotopic vision in dark conditions, and mesopic vision between these two. A computational model is designed to simulate multiple features of mesopic vision and scotopic vision, including the chromaticity change, luminance change, and visual acuity loss. The system uses a source image under bright illumination as input. Then assuming that the viewer has already adapted to the new conditions, the color spectrum of the input image is reconstructed to replace the source with modifications of the chromaticity and the luminance of the relighted scene. A bilateral filter is used to simulate the visual acuity loss. The model parameters have clear physical meanings and can be obtained from experimental data to achieve realistic results.The algorithm can be used not only for visual perception simulation, but also as a day-for-night tool to produce realistic nighttime images from daytime images.

International audience; This paper presents a novel content-based method for transferring the colour patterns between images. Unlike previous methods that rely on image colour statistics, our method puts an emphasis on high-level scene content analysis. We first automatically extract the foreground subject areas and background scene layout from the scene. The semantic correspondences of the regions between source and target images are established. In the second step, the source image is re-coloured in a novel optimization framework, which incorporates the extracted content information and the spatial distributions of the target colour styles. A new progressive transfer scheme is proposed to integrate the advantages of both global and local transfer algorithms, as well as avoid the over-segmentation artefact in the result. Experiments how that with a better understanding of the scene contents, our method well preserves the spatial layout, the colour distribution and the visual coherence in the transfer process. As an interesting extension, our method can also be used to re-colour video clips with spatially-varied colour effects.

International audience; Point cloud is a basic description of discrete shape information. Parameterization of unorganized points is important for shape analysis and shape reconstruction of natural objects. In this paper we present a new algorithm for global parameterization of an unorganized point cloud and its application to the meshing of the cloud. Our method is guided by principal directions so as to preserve the intrinsic geometric properties. After initial estimation of principal directions, we develop a kNN(k-nearest neighbor) graph-based method to get a smooth direction field. Then the point cloud is cut to be topologically equivalent to a disk. The global parameterization is computed and its gradients align well with the guided direction field. A mixed integer solver is used to guarantee a seamless parameterization across the cut lines. The resultant parameterization can be used to triangulate and quadrangulate the point cloud simultaneously in a fully automatic manner, where the shape of the data is of any genus.

International audience; We present a novel method for content-aware image resizing based on optimization of a well-defined image distance function, which preserves both the important regions and the global visual effect (the background or other decorative objects) of an image. The method operates by joint use of seam carving and image scaling. The principle behind our method is the use of a bidirectional similarity function of image Euclidean distance (IMED), while cooperating with a dominant color descriptor (DCD) similarity and seam energy variation. The function is suitable for the quantitative evaluation of the resizing result and the determination of the best seam carving number. ifferent from the previous simplex-modeapproaches, our method takes the advantages of both discrete and continuous methods. The technique is useful in image resizing for both reduction/retargeting and enlarging. We also show that this approach can be extended to indirect image resizing.

The original publication is avalaible at www.springerlink.com; International audience; One significant problem in tile-based texture synthesis is the presence of conspicuous seams in the tiles. The reason is that sample patches employed as primary patterns of the tile set may not be well stitched if carelessly picked. In this paper, we introduce a robust approach that can stably generate an ω-tile set of high quality and pattern diversity. First, an extendable rule is introduced to increase the number of sample patches to vary the patterns in an ω-tile set. Second, in contrast to other concurrent techniques that randomly choose sample patches for tile construction, ours uses artificial immune system (AIS) to select the feasible patches from the input example. This operation ensures the quality of the whole tile set. Experimental results verify the high quality and efficiency of the proposed algorithm.

The original publication is available at www.springerlink.com; International audience; This paper presents a novel texture synthesis scheme for anisotropic 2D textures based on perspective feature analysis and energy optimization. Given an example texture, the synthesis process starts with analyzing the texel (TEXture ELement) scale variations to obtain the perspective map (scale map). Feature mask and simple user-assisted scale extraction operations including slant and tilt angles assignment and scale value editing are applied. The scale map represents the global variations of the texel scales in the sample texture. Then, we extend 2D texture optimization techniques to synthesize these kinds of perspectively featured textures. The non-parametric texture optimization approach is integrated with histogram matching, which forces the global statics of the texel scale variations of the synthesized texture to match those of the example. We also demonstrate that our method is well-suited for image completion of a perspectively featured texture region in a digital photo.

International audience; One significant problem in tile-based texture synthesis is the presence of conspicuous seams in the tiles. The reason is that the sample patches employed as primary patterns of the tile set may not be well stitched if carelessly picked. In this paper, we introduce an optimized approach that can stably generate an !-tile set of high pattern diversity and high quality. Firstly, an extendable rule is introduced to increase the number of sample patches to vary the patterns in an ω-tile set. Secondly, in contrast to the other concurrent techniques that randomly choose sample patches for tile construction, our technique uses Genetic Algorithm to select the feasible patches from the input example. This operation insures the quality of the whole tile set. Experimental results verify the high quality and efficiency of the proposed algorithm.

http://www.springerlink.com/; Rendering the structural color of natural objects or modern industrial products in the 3D environment is not possible with RGB-based graphics platforms and software and very time consuming, even with the most efficient spectra representation based methods previously proposed. Our framework allows computing full spectra light object interactions only when it is needed, i.e. for the part of the scene that requires simulating special spectra sensitive phenomena. Achieving the rendering of complex scenes with both the full spectra and RGB light and object interactions in a ray-tracer costs only some additional fractions of seconds. To prove the efficiency of our framework, we implemented a “Multilayer Film” in a simple ray-tracer. However, the framework is convenient for any complex lighting model, including diffraction or fluorescence.

International audience; This paper presents a biologically-motivated system of seasonal variant scenes generation for maples, which has a obvious leaf color transformation during the time. Given climate data and knowledge on environmental influence to maples, our system is able to simulate this seasonal leaf color transformation process. Our system consists of three steps: environment configuration, climate influence simulation and leaf texture acquisition. The first step decides the general color change timing of the maple tree based on its local environment. Then we make further adjustments to the timing determined in the last step taking into account the influence of climate in the specific case. In the last step, the texture maps of leaves are generated based on the pigment information. Our system is also able to simulate the seasonal color variance of other trees by adjusting related parameters.

International audience; One significant problem in tile-based texture synthesis is the selection of sample patches. The reason is that the tiling process may produce results with poor quality when the sample patches used to fill the tiles can not find good cutting pathes at the merging stage. In this paper, we use Genetic Algorithm to find the best group of sample patches from a considerable number of patch groups. Using tile set gene rated by these optimal sample patches can significantly improve the result quality of the tile-based texture synthesis.