1. ROAM: a rich object appearance model with application to rotoscoping
- Author
-
Juan-Manuel Perez-Rua, Patrick Bouthemy, Philip H. S. Torr, Tomas Crivelli, Patrick Pérez, Ondrej Miksik, University of Oxford, Orange Labs R&D [Rennes], France Télécom, Technicolor R & I [Cesson Sévigné], Technicolor, Space-timE RePresentation, Imaging and cellular dynamics of molecular COmplexes (SERPICO), Inria Rennes – Bretagne Atlantique, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), and Institut National de Recherche en Informatique et en Automatique (Inria)
- Subjects
Structure (mathematical logic) ,Computer science ,business.industry ,Applied Mathematics ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,[INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV] ,Initialization ,02 engineering and technology ,Object (computer science) ,Active appearance model ,Computational Theory and Mathematics ,Artificial Intelligence ,Compositing ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Segmentation ,Computer vision ,Computer Vision and Pattern Recognition ,Artificial intelligence ,business ,ComputingMilieux_MISCELLANEOUS ,Software - Abstract
Rotoscoping , the detailed delineation of scene elements through a video shot, is a painstaking task of tremendous importance in professional post-production pipelines. While pixel-wise segmentation techniques can help for this task, professional rotoscoping tools rely on parametric curves that offer the artists a much better interactive control on the definition, editing and manipulation of the segments of interest. Sticking to this prevalent rotoscoping paradigm, we propose a novel framework to capture and track the visual aspect of an arbitrary object in a scene, given an initial closed outline of this object. This model combines a collection of local foreground/background appearance models spread along the outline, a global appearance model of the enclosed object and a set of distinctive foreground landmarks. The structure of this rich appearance model allows simple initialization, efficient iterative optimization with exact minimization at each step, and on-line adaptation in videos. We further extend this model by so-called trimaps which serve as an input to alpha-matting algorithms to allow truly seamless compositing. To this end, we leverage local classifiers attached to the roto-curves to define a confidence measure that is well-suited to define trimaps with adaptive band-widths. The resulting trimaps are parametric, temporally consistent and remain fully editable by the artist. We demonstrate qualitatively and quantitatively the merit of this framework through comparisons with tools based on either dynamic segmentation with a closed curve or pixel-wise binary labelling.
- Published
- 2020