Search

Green screen keying has always been an essential and fundamental part of film and television special effects. In the actual shooting process, captured green screen images vary significantly due to the comprehensive influence of lighting, shooting angle, green cloth material, characters, etc. In order to obtain visually pleasing effects, traditional methods usually require professionals to adjust the corresponding parameters with lots of workload for different images, which is inefficient. Meanwhile, there are tedious steps in dealing with the green spill problem. In this paper, we propose a deep learning-based green screen keying method which is automatic, effective, and real-time. Firstly, we create a green screen dataset that contains not only alpha and foreground maps but also samples with green spill phenomenon and a large number of distinct green screen backgrounds. Secondly, we propose an end-to-end network that can automatically tackle green screen keying and green spill removal problems. Our method only takes a single image as input without user interaction and estimates alpha and foreground simultaneously. Extensive experiments clearly demonstrate the superiority of our proposed method. Moreover, our method achieves approximately 75fps on 720P videos (1280 × 720) and 25fps on 1080P videos (1920 × 1080), which can be considered real-time for many applications. [ABSTRACT FROM AUTHOR]

Image style transfer has attracted much attention in recent years. However, results produced by existing works still have lots of distortions. This paper investigates the CNN-based artistic style transfer work specifically and finds out the key reasons for distortion coming from twofold: the loss of spatial structures of content image during content-preserving process and unexpected geometric matching introduced by style transformation process. To tackle this problem, this paper proposes a novel approach consisting of a dual-stream deep convolution network as the loss network and edge-preserving filters as the style fusion model. Our key contribution is the introduction of an additional similarity loss function that constrains both the detail reconstruction and style transfer procedures. The qualitative evaluation shows that our approach successfully suppresses the distortions as well as obtains faithful stylized results compared to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The event camera is a kind of visual sensor that mimics aspects of the human visual system by only recording events when the light intensity on a pixel changes. This allows for an event camera to possess high temporal resolution and makes it able to capture fast motion. However, an event camera lacks information for all pixels within a scene, especially color information. In this paper, we aim to recover a typical scene in which the foreground undergoes high-speed motion which can be approximated by a planar motion, and the background is static. We demonstrate how to use the event camera to generate high-speed videos of 2D motion augmented with foreground and background images taken from a conventional camera. We match an object obtained for a static image to frames formed by the event stream, from the event camera, based on curve saliency, and we build a parametric model of affine motion to create image sequences. In this work, we are able to restore scenes of very fast motion such as falling or rotating objects and string vibration. [ABSTRACT FROM AUTHOR]

Numerous depth image-based rendering algorithms have been proposed to synthesize the virtual view for the free viewpoint television. However, inaccuracies in the depth map cause visual artifacts in the virtual view. In this paper, we propose a novel virtual view synthesis framework to create the virtual view of the scene. Here, we incorporate a trilateral depth filter with local texture information, spatial proximity, and color similarity to remove the ghost contours by rectifying the misalignment between the depth map and its associated color image. To further enhance the quality of the synthesized virtual views, we partition the scene into different 3D object segments based on the color image and depth map. Each 3D object segment is warped and blended independently to avoid mixing the pixels belonging to different parts of the scene. The evaluation results indicate that the proposed method significantly improves the quality of the synthesized virtual view compared with other methods and are qualitatively very similar to the ground truth. In addition, it also performs well in real-world scenes. [ABSTRACT FROM AUTHOR]

This paper presents a fast smoothed particle hydro-dynamics (SPH) simulation approach for gaseous fluids. Unlike previous SPH gas simulators, which solve the transparent air flow in a fixed simulation domain, the proposed approach directly solves the visible gas without involving the transparent air. By compensating the density and force calculation for the visible gas particles, we completely avoid the need of computational cost on ambient air particles in previous approaches. This allows the computational resources to be exclusively focused on the visible gas, leading to significant performance improvement of SPH gas simulation. The proposed approach is at least ten times faster than the standard SPH gas simulation strategy and is able to reduce the total particle number by 25-400 times in large open scenes. The proposed approach also enables fast SPH simulation of complex scenes involving liquid-gas transition, such as boiling and evaporation. A particle splitting and merging scheme is proposed to handle the degraded resolution in liquid-gas phase transition. Various examples are provided to demonstrate the effectiveness and efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

We propose a novel depth perception model to determine the time taken by the human visual system (HVS) to adapt to an abrupt change in stereoscopic disparity, such as can occur in a scene cut. A series of carefully designed perceptual experiments on successive disparity contrast were used to build our model. Factors such as disparity, changes in disparity, and the spatial frequency of luminance contrast were taken into account. We further give a computational method to predict the response time during scene cuts in stereoscopic cinematography, which has been validated in user studies. We also consider various applications of our model. [ABSTRACT FROM AUTHOR]

In this paper, we have proposed a novel patch-based method for automatic completion of stereoscopic images and the corresponding depth/disparity maps simultaneously. The missing depths are estimated in local feature space and a patch distance metric is designed to take the appearance, depth gradients and depth inconsistency into account. To ensure the proper stereopsis, we first search for the proper stereoscopic patch in both left and right images according to the distance metric, and then iteratively refine the images. Our method is capable of dealing with general scenes including both frontal-parallel and non-frontal-parallel objects. Experimental results show that our method is superior to previous ones with better stereoscopically consistent content and more plausible completion. [ABSTRACT FROM AUTHOR]

We present a method for learning a meta-filter from an example pair comprising an original image $$A$$ and its filtered version $$A^\prime $$ using an unknown image filter. A meta-filter is a parametric model, consisting of a spatially varying linear combination of simple basis filters. We introduce a technique for learning the parameters of the meta-filter $$f$$ such that it approximates the effects of the unknown filter, i.e., $$f(A)$$ approximates $$A^\prime $$ . The meta-filter can be transferred to novel input images, and its parametric representation enables intuitive tuning of its parameters to achieve controlled variations. We show that our technique successfully learns and models meta-filters that approximate a large variety of common image filters with high accuracy both visually and quantitatively. [ABSTRACT FROM AUTHOR]

Efficiently identifying salient objects in large image collections is essential for many applications including image retrieval, surveillance, image annotation, and object recognition. We propose a simple, fast, and effective algorithm for locating and segmenting salient objects by analysing image collections. As a key novelty, we introduce group saliency to achieve superior unsupervised salient object segmentation by extracting salient objects (in collections of pre-filtered images) that maximize between-image similarities and within-image distinctness. To evaluate our method, we construct a large benchmark dataset consisting of 15 K images across multiple categories with 6000+ pixel-accurate ground truth annotations for salient object regions where applicable. In all our tests, group saliency consistently outperforms state-of-the-art single-image saliency algorithms, resulting in both higher precision and better recall. Our algorithm successfully handles image collections, of an order larger than any existing benchmark datasets, consisting of diverse and heterogeneous images from various internet sources. [ABSTRACT FROM AUTHOR]

In recent years, the computer graphics and computer vision communities have devoted significant attention to research based on Internet visual media resources. The huge number of images and videos continually being uploaded by millions of people have stimulated a variety of visual media creation and editing applications, while also posing serious challenges of retrieval, organization, and utilization. This article surveys recent research as regards processing of large collections of images and video, including work on analysis, manipulation, and synthesis. It discusses the problems involved, and suggests possible future directions in this emerging research area. [ABSTRACT FROM AUTHOR]

Abstract  This paper considers the problem of placing mosaic tiles on a surface to produce a surface mosaic. We assume that the user specifies a mesh model, the size of the tiles and the amount of grout, and, optionally, a few control vectors at key locations on the surface indicating the preferred tile orientation at these points. From these inputs, we place equal-sized rectangular tiles over the mesh such as to almost cover it, with controlled orientation. The alignment of the tiles follows a vector field which is interpolated over the surface from the control vectors and also forced into alignment with any sharp creases, open boundaries, and boundaries between regions of different colors. Our method efficiently solves the problem by posing it as globally optimizing a spring-like energy in the Manhattan metric, using overlapping local parameterizations. We demonstrate the effectiveness of our algorithm with various examples. [ABSTRACT FROM AUTHOR]

Abstract Video completion is the problem of automatically filling space–time holes in video sequences left by the removal of unwanted objects in a scene. We solve it using texture synthesis, filling a hole inwards using three steps iteratively: we select the most promising target pixel at the edge of the hole, we find the source fragment most similar to the known part of the target’s neighborhood, and we merge source and target fragments to complete the target neighborhood, reducing the size of the hole. [ABSTRACT FROM AUTHOR]