Search

Perfect image compositing can harmonize the appearance between the foreground and background effectively so that the composite result looks seamless and natural. However, the traditional convolutional neural network (CNN)-based methods often fail to yield highly realistic composite results due to overdependence on scene parsing while ignoring the coherence of semantic and structural between foreground and background. In this paper, we propose a framework to solve this problem by training a stacked generative adversarial network with attention guidance, which can efficiently create a high-resolution, realistic-looking composite. To this end, we develop a diverse adversarial loss in addition to perceptual and guidance loss to train the proposed generative network. Moreover, we construct a multi-scenario dataset for high-resolution image compositing, which contains high-quality images with different styles and object masks. Experiments on the synthesized and real images demonstrate the efficiency and effectiveness of our network in producing seamless, natural, and realistic results. Ablation studies show that our proposed network can improve the visual performance of composite results compared with the application of existing methods.

Feature engineering usually needs to excavate dense-and-implicit cross features from multi-filed sparse data. Recently, many state-of-the-art models have been proposed to achieve low-order and high-order feature interactions. However, most of them ignore the importance of cross features and fail to suppress the negative impact of useless features. In this paper, a novel multi-scale feature-crossing attention network (MsFcNET) is proposed to extract dense-and-implicit cross features and learn their importance in the different scales. The model adopts the DIA-LSTM units to construct a new attention calibration architecture, which can adaptively adjust the weights of features in the process of feature interactions. On the other hand, it also integrates a multi-scale feature-crossing module to strengthen the representation ability of cross features from multi-field sparse data. The extensive experimental results on three real-world prediction datasets demonstrate that our proposed model yields superior performance compared with the other state-of-the-art models.

A number of edge-aware filters can efficiently boost the appearance of an image by detail decomposition and enhancement. However, they often fail to produce photographic enhanced appearance due to some visible artifacts, especially noise, halos and unnatural contrast. The essential reason is that the guidance and the constraint of high-quality appearance are not sufficient enough in the process of enhancement. Thus our idea is to train a detail dictionary from a lot of high-quality patches in order to constrain and control the entire appearance enhancement. In this paper, we propose a novel learningbased enhancement method for photographic appearance, which includes two main stages: dictionary training and sparse reconstruction. In the training stage, we construct a training set of detail patches extracted from some high-quality photos, and then train an overcomplete detail dictionary by iteratively minimizing an l 1-norm energy function. In the reconstruction stage, we employ the trained dictionary to reconstruct the boosted detail layer, and further formalize a gradient-guided optimization function to improve the local coherence between patches. Moreover, we propose two evaluation metrics to measure the performance of appearance enhancement. The final experimental results have demonstrated the effectiveness of our learning-based enhancement method.

Cigarette code is a string printed on the wrapper of cigarette packet as a basis of distinguishing illegal sales for tobacco administration. In general, the code is excerpted and entered to administration system manually during on-site inspection, which is quite time-consuming and laborious. In this paper, we propose a new solution based on convolutional neural network for intelligent transcription. Our recognition method is composed of four components: detection, identification, alignment, and regularization. First of all, the detection component fine-tunes an end-to-end detection network to obtain the bounding box region of cigarette code. Then the identification component constructs an optimized CNN architecture to recognize each character in the region of cigarette code. Meanwhile the alignment component trains a CPM-based network to estimate the positions of all characters including some missing characters. Finally, the regularization component develops a matching algorithm to produce a regularized result with all characters. The experimental results demonstrate that our proposed method can perform a better, faster and more labor-saving cigarette code transcription process.

Face aging is one of research hotspots in the field of computer vision, but there is no mature system yet that can be applied to digital entertainment. This paper proposes a face aging system based on feature fusion, which can synthesize aging facial images of young people efficiently and quickly by using facial texture features of elderly people's facial images, achieve the goal of simulating the appearance after aging. The system published online by the “eleme” app for the Mother's Day, showing great potential for digital entertainment.

Face anti-spoofing, as a security measure for face verification and recognition system, could distinguish between genuine and fake faces. Although some impressive results have been achieved by CNN when evaluated on intra-tests (i.e. the model is trained and tested on the same dataset). Unfortunately, most of these models fail to generalize well to unseen attacks (e.g. when the model is trained on one dataset and then evaluated on another dataset). This is a major concern in face anti-spoofing research which is mostly overlooked. Our experiments on two challenging benchmark face spoofing datasets, CASIA and Replay-Attack, demonstrate the disappointed adaptation ability for CNN from one dataset to another. Via visualizing the implicit attention of CNN, we find that scene-dependent features extracted by CNN affect models generalization capabilities. To address this problem, we propose a novel solution based on applying scene-independent features representation.

Video facial swap usually has strong entertainment applications, and it is also applicable for the post-production of films and has great application value. At present, the popular face swap is done manually by the PS software, and the synthetic effect of the automatic face changing technology is not good. In order to make up for the lack of these features, this paper proposes a method of video face swap based on autoencoder generation network. The network learns the mapping relationship between distorted face and original face: the encoder can distinguish and extract facial information, and the decoder can restore face separately. First, the local information of tow face is sent to the network to get the initial model; then, the global information is put into the network for fine-tuning; finally, the face exchange between A and B is completed with face alignment and alpha fusion. The experimental results show that the quality of the method is improved significantly.

High dynamic range (HDR) imaging has seen a lot of progress in recent years, while an efficient way to capture and generate HDR video is still in need. In this paper, we present a method to generate HDR video from frame sequence of alternating exposures in a fast and concise fashion. It takes advantage of the recent advancement in deep learning to achieve superior efficiency compared to other state-of-art method. By training an end-to-end CNN model to estimate optical flow between frames of different exposures, we are able to achieve dense image registration of them. Using this as a base, we develop an efficient method to reconstruct the aligned LDR frames with different exposure and then merge them to produce the corresponding HDR frame. Our approach shows good performance and time efficiency while still maintain a relatively concise framework.

Tone mapping can compress the color range of HDR (high dynamic range) image to produce its LDR (low dynamic range) image. However, without preserving the original scene detail, tone mapping may produce various artifacts and their fidelity will be severely decreased. In this paper, we propose a new detail-preserving refinement method to efficiently improve the fidelity of tone mapping. Its overall flow includes the four key steps: tone initialization, detail extraction, tone reconstruction, and detail restoration. First of all, we employ clustering and decomposition to obtain initial tone layer and detail map. Then we construct detail affinity and gradient guidance to optimize the detail map and reconstruct the new tone layer. Finally, we combine them to produce a high-fidelity result of tone mapping. We demonstrate the effectiveness of the proposed method through a number of experiments in visual comparison, and verify its high-efficiency performance in execution time.

In this paper, a real-time facial animation system is presented, that captures live facial expressions and poses from user, and uses them to animate a synthetic 2.5D cartoon character (Avatar) in real-time. Compared to existing 3D Avatar systems, the 2.5D cartoon models are easier to create than full 3D model, and the cartoon-style drawings are more popular among young people. Our system requires only an ordinary PC camera, and the initialization is fully automatic. We established a mapping from real-time facial motion to the atomic motion parameters of the cartoon model by pose and expression estimation. The animation is driven by these parameters in the way of blend shape interpolation.

Generally, for a global or local color transfer, the traditional approaches will rearrange color distribution in source image according to reference image. However, the destruction of scene detail and illumination environment might produce a low-fidelity color transfer result. In this paper, we propose a unified fidelity optimization model for color transfer to yield a high-fidelity transfer result in terms of color, detail and illumination. Corresponding to the three characteristics, our approach is described as an optimization problem with three energy terms: color mapping, detail preserving and illumination awareness. Color mapping can employ histogram matching to impose the color style of reference image on source image; Detail preserving can apply gradient guidance to maintain scene detail in source image; Illumination awareness can construct illumination affinity to harmonize illumination environment. Moreover, following the definition of fidelity with three characteristics, we also propose an objective evaluation metric to analyze the performance of our approach in different coefficients. The comparison of experiment results demonstrates the effectiveness of our optimization model.