Search

Sketch-based modeling strives to bring the ease and immediacy of drawing to the 3D world. However, while drawings are easy for humans to create, they are very challenging for computers to interpret due to their sparsity and ambiguity. We propose a data-driven approach that tackles this challenge by learning to reconstruct 3D shapes from one or more drawings. At the core of our approach is a deep convolutional neural network (CNN) that predicts occupancy of a voxel grid from a line drawing. This CNN provides us with an initial 3D reconstruction as soon as the user completes a single drawing of the desired shape. We complement this single-view network with an updater CNN that refines an existing prediction given a new drawing of the shape created from a novel viewpoint. A key advantage of our approach is that we can apply the updater iteratively to fuse information from an arbitrary number of viewpoints, without requiring explicit stroke correspondences between the drawings. We train both CNNs by rendering synthetic contour drawings from hand-modeled shape collections as well as from procedurally-generated abstract shapes. Finally, we integrate our CNNs in a minimal modeling interface that allows users to seamlessly draw an object, rotate it to see its 3D reconstruction, and refine it by re-drawing from another vantage point using the 3D reconstruction as guidance. The main strengths of our approach are its robustness to freehand bitmap drawings, its ability to adapt to different object categories, and the continuum it offers between single-view and multi-view sketch-based modeling., Comment: See our accompanying video on https://youtu.be/DGIYzmlm2pQ, networks and databases available at https://ns.inria.fr/d3/3DSketching/. To appear in PACMCGIT

International audience; Creating a virtual city is demanded for computer games, movies, and urban planning, but it takes a lot of time to create numerous 3D building models. Procedural modeling has become popular in recent years to overcome this issue, but creating a grammar to get a desired output is difficult and time consuming even for expert users. In this paper, we present an interactive tool that allows users to automatically generate such a grammar from a single image of a building. The user selects a photograph and highlights the silhouette of the target building as input to our method. Our pipeline automatically generates the building components, from large-scale building mass to fine-scale windows and doors geometry. Each stage of our pipeline combines convolutional neural networks (CNNs) and optimization to select and parameterize procedural grammars that reproduce the building elements of the picture. In the first stage, our method jointly estimates camera parameters and building mass shape. Once known, the building mass enables the rectification of the façades, which are given as input to the second stage that recovers the façade layout. This layout allows us to extract individual windows and doors that are subsequently fed to the last stage of the pipeline that selects procedural grammars for windows and doors. Finally, the grammars are combined to generate a complete procedural building as output. We devise a common methodology to make each stage of this pipeline tractable. This methodology consists in simplifying the input image to match the visual appearance of synthetic training data, and in using optimization to refine the parameters estimated by CNNs. We used our method to generate a variety of procedural models of buildings from existing photographs.

Solving linear system of equations stemming from Laplacian operators is at the heart of a wide range of applications. Due to the sparsity of the linear systems, iterative solvers such as Conjugate Gradient and Multigrid are usually employed when the solution has a large number of degrees of freedom. These iterative solvers can be seen as sparse approximations of the Green's function for the Laplacian operator. In this paper we propose a machine learning approach that regresses a Green's function from boundary conditions. This is enabled by a Green's function that can be effectively represented in a multi-scale fashion, drastically reducing the cost associated with a dense matrix representation. Additionally, since the Green's function is solely dependent on boundary conditions, training the proposed neural network does not require sampling the right-hand side of the linear system. We show results that our method outperforms state of the art Conjugate Gradient and Multigrid methods., Computers & Graphics, 107, ISSN:0097-8493, ISSN:1873-7684

International audience; We present an interactive system to ease the creation of so-called "video doodles" -- videos on which artists insert hand-drawn animations for entertainment or educational purposes. Video doodles are challenging to create because to be convincing, the inserted drawings must appear as if they were part of the captured scene. In particular, the drawings should undergo tracking, perspective deformations and occlusions as they move with respect to the camera and to other objects in the scene -- visual effects that are difficult to reproduce with existing 2D video editing software. Our system supports these effects by relying on planar canvases that users position in a 3D scene reconstructed from the video. Furthermore, we present a custom tracking algorithm that allows users to anchor canvases to static or dynamic objects in the scene, such that the canvases move and rotate to follow the position and direction of these objects. When testing our system, novices could create a variety of short animated clips in a dozen of minutes, while professionals praised its speed and ease of use compared to existing tools.

International audience; Radiance Field methods have recently revolutionized novel-view synthesisof scenes captured with multiple photos or videos. However, achieving highvisual quality still requires neural networks that are costly to train and render,while recent faster methods inevitably trade off speed for quality. Forunbounded and complete scenes (rather than isolated objects) and 1080presolution rendering, no current method can achieve real-time display rates.We introduce three key elements that allow us to achieve state-of-the-artvisual quality while maintaining competitive training times and importantlyallow high-quality real-time (≥ 30 fps) novel-view synthesis at 1080p resolution.First, starting from sparse points produced during camera calibration,we represent the scene with 3D Gaussians that preserve desirable propertiesof continuous volumetric radiance fields for scene optimization whileavoiding unnecessary computation in empty space; Second, we performinterleaved optimization/density control of the 3D Gaussians, notably optimizinganisotropic covariance to achieve an accurate representation of thescene; Third, we develop a fast visibility-aware rendering algorithm thatsupports anisotropic splatting and both accelerates training and allows realtimerendering. We demonstrate state-of-the-art visual quality and real-timerendering on several established datasets.

CAD modeling, despite being the industry-standard, remains restricted to usage by skilled practitioners due to two key barriers. First, the user must be able to mentally parse a final shape into a valid sequence of supported CAD commands; and second, the user must be sufficiently conversant with CAD software packages to be able to execute the corresponding CAD commands. As a step towards addressing both these challenges, we present Free2CAD wherein the user can simply sketch the final shape and our system parses the input strokes into a sequence of commands expressed in a simplified CAD language. When executed, these commands reproduce the sketched object. Technically, we cast sketch-based CAD modeling as a sequence-to-sequence translation problem, for which we leverage the powerful Transformers neural network architecture. Given the sequence of pen strokes as input, we introduce the new task of grouping strokes that correspond to individual CAD operations. We combine stroke grouping with geometric fitting of the operation parameters, such that intermediate groups are geometrically corrected before being reused, as context, for subsequent steps in the sequence inference. Although trained on synthetically-generated data, we demonstrate that Free2CAD generalizes to sketches created from real-world CAD models as well as to sketches drawn by novice users. Code and data are at https://github.com/Enigma-li/Free2CAD.

International audience; Neural Radiance Fields (NeRFs) have revolutionized novel view synthesis for captured scenes, with recent methods allowing interactive free-viewpoint navigation and fast training for scene reconstruction. However, the implicit representations used by these methods — often including neural networks and complex encodings— make them difficult to edit. Some initial methods have been proposed, but they suffer from limited editing capabilities and/or from a lack of interactivity, and are thus unsuitable for interactive editing of captured scenes. We tackle both limitations and introduce NeRFshop, a novel end-to-end method that allows users to interactively select and deform objects through cage-based transformations. NeRFshop provides fine scribblebaseduser control for the selection of regions or objects to edit, semi-automatic cage creation, and interactive volumetric manipulation of scene content thanks to our GPU-friendly two-level interpolation scheme. Further, we introduce a preliminary approach that reduces potential resulting artifacts of these transformations with a volumetric membrane interpolation technique inspired by Poisson image editing and provide a process that “distills” the edits into a standalone NeRF representation.

International audience; We propose an interactive animation method for the ejection of gas and ashes mixtures in volcano eruption. Our novel, layered solution combines a coarse-grain, physically-based simulation of the ejection dynamics with a consistent, procedural animation of multi-resolution details. We show that this layered model can be used to capture the two main types of ejection, namely ascending plume columns composed of rapidly rising gas carrying ash which progressively entrains more air, and pyroclastic flows which descend the slopes of the volcano depositing ash, ultimately leading to smaller plumes along their way.We validate the large-scale consistency of our model through comparison with geoscience data, and discuss both real-time visualization and off-line, realistic rendering.

OPAL-Meso; International audience; Recent advances in neural rendering indicate immense promise for architectures that learn light transport, allowing efficient rendering of global illumination effects once such methods are trained. The training phase of these methods can be seen as a form of pre-computation, which has a long standing history in Computer Graphics. In particular, Pre-computed Radiance Transfer (PRT) achieves real-time rendering by freezing some variables of the scene (geometry, materials) and encoding the distribution of others, allowing interactive rendering at runtime. We adopt the same configuration as PRT – global illumination of static scenes under dynamic environment lighting – and investigate different neural network architectures, inspired by the design principles and theoretical analysis of PRT. We introduce four different architectures, and show that those based on knowledge of light transport models and PRT-inspired principles improve the quality of global illumination predictions at equal training time and network size, without the need for high-end ray-tracing hardware.

International audience; We introduce a new mechanism for self-actuating deployable structures, based on printing a dense pattern of closely-spaced plastic ribbons on sheets of pre-stretched elastic fabric. We leverage two shape-changing effects that occur when such an assembly is printed and allowed to relax: first, the incompressible plastic ribbons frustrate the contraction of the fabric back to its rest state, forcing residual strain in the fabric and creating intrinsic curvature. Second, the differential compression at the interface between the plastic and fabric layers yields a bilayer effect in the direction of the ribbons, making each ribbon buckle into an arc at equilibrium state and creating extrinsic curvature. We describe an inverse design tool to fabricate low-cost, lightweight prototypes of freeform surfaces using the controllable directional distortion and curvature offered by this mechanism. The core of our method is a parameterization algorithm that bounds surface distortions along and across principal curvature directions, along with a pattern synthesis algorithm that covers a surface with ribbons to match the target distortions and curvature given by the aforementioned parameterization. We demonstrate the flexibility and accuracy of our method by fabricating and measuring a variety of surfaces, including nearly-developable surfaces as well as surfaces with positive and negative mean curvature, which we achieve thanks to a simple hardware setup that allows printing on both sides of the fabric.

International audience; The design of car shapes requires a delicate balance between aesthetic and performance. While fluid simulation provides themeans to evaluate the aerodynamic performance of a given shape, its computational cost hinders its usage during the early explorative phases of design, when aesthetic is decided upon. We present an interactive system to assist designers in creating aerodynamic car profiles. Our system relies on a neural surrogate model to predict fluid flow around car shapes, providing fluid visualization and shape optimization feedback to designers as soon as they sketch a car profile. Compared to prior work that focused on time-averaged fluid flows, we describe how to train our model on instantaneous, synchronized observations extracted from multiple pre-computed simulations, such that we can visualize and optimize for dynamic flow features, such as vortices. Furthermore, we architectured our model to support gradient-based shape optimization within a learned latent space of car profiles. In addition to regularizing the optimization process, this latent space and an associated encoder-decoder allows us to input and output car profiles in a bitmap form, without any explicit parameterization of the car boundary. Finally, we designed our model to support pointwise queries of fluid properties around car shapes, allowing us to adapt computational cost to application needs. As an illustration, we only query our model along streamlines for flow visualization, we query it in the vicinity of the car for drag optimization, and we query it behind the car for vortex attenuation.

View-dependent effects such as reflections pose a substantial challenge for image-based and neural rendering algorithms. Above all, curved reflectors are particularly hard, as they lead to highly non-linear reflection flows as the camera moves. We introduce a new point-based representation to compute Neural Point Catacaustics allowing novel-view synthesis of scenes with curved reflectors, from a set of casually-captured input photos. At the core of our method is a neural warp field that models catacaustic trajectories of reflections, so complex specular effects can be rendered using efficient point splatting in conjunction with a neural renderer. One of our key contributions is the explicit representation of reflections with a reflection point cloud which is displaced by the neural warp field, and a primary point cloud which is optimized to represent the rest of the scene. After a short manual annotation step, our approach allows interactive high-quality renderings of novel views with accurate reflection flow. Additionally, the explicit representation of reflection flow supports several forms of scene manipulation in captured scenes, such as reflection editing, cloning of specular objects, reflection tracking across views, and comfortable stereo viewing. We provide the source code and other supplemental material on https://repo-sam.inria.fr/ fungraph/neural_catacaustics/, Comment: SIGGRAPH Asia 2022 (ToG) https://repo-sam.inria.fr/fungraph/neural_catacaustics/

International audience; Recent advances in Neural Radiance Fields enable the capture of scenes with motion. However, editing the motion is hard; no existing method allows editing beyond the space of motion existing in the original video, nor editing based on physics. We present the first approach that allows physically-based editing of motion in a scene captured with a single hand-held video camera, containing vibrating or periodic motion. We first introduce a Lagrangian representation, representing motion as the displacement of particles, which is learned while training a radiance field. We use these particles to create a continuous representation of motion over the sequence, which is then used to perform a modal analysis of the motion thanks to a Fourier transform on the particle displacement over time. The resulting extracted modes allow motion synthesis, and easy editing of the motion, while inheriting the ability for free-viewpoint synthesis in the captured 3D scene from the radiance field. We demonstrate our new method on synthetic and real captured scenes.

Computer graphics strives to render synthetic images identical to real photographs. Multiple rendering algorithms have been developed for the better part of the last half-century. Traditional algorithms use 3D assets manually generated by artists to render a scene. While the initial scenes were quite simple, the field has developed complex representations of geometry, material and lighting: the three basic components of a 3D scene. Generating such complex assets is hard and requires significant time and skills by professional 3D artists. In addition to asset generation, the rendering algorithms themselves involve complex simulation techniques to solve for global light transport in a scene which costs more time.As the ease of capturing photographs improved, Image-based Rendering (IBR) emerged as an alternative to traditional rendering. Using captured images as input became much faster than generating traditional scene assets. Initial IBR algorithms focused on creating a scene model using the input images to interpolate or warp them and enable free-viewpoint navigation of captured scenes. With time the scene models became more complex and using a geometric proxy computed from the input images became an integral part of IBR. Today using a mesh reconstructed using Structure-from-Motion (SfM) and Multi-view Stereo (MVS) techniques is widely used in IBR even though they introduce significant artifacts due to noisy reconstruction.In this thesis we first propose a novel image-based rendering algorithm, which focuses on rendering a captured scene with good quality at interactive frame rates}. We study different artifacts from previous IBR algorithms and propose an algorithm which builds upon previous work to remove such artifacts. The algorithm utilizes surface appearance in order to treat view-dependent regions differently than diffuse regions. Our Hybrid-IBR algorithm performs favorably against classical and modern IBR approaches for a wide variety of scenes in terms of quality and/or speed.While IBR provides solutions to render a scene, editing them is hard. Editing scenes require estimating a scene's geometry, material appearance and illumination. As our second contribution \textbf{we explicitly estimate \emph{scene-scale} material parameters from a set of captured photographs to enable scene editing}. While commercial photogrammetry solutions recover diffuse texture to aid 3D artists in generating material assets manually, we aim to \emph{automatically} create material texture atlases from captured images of a scene. We take advantage of the visual cues provided by the multi-view observations. Feeding it to a Convolutional Neural Network (CNN) we obtain material maps for each view. Using the predicted maps we create multi-view consistent material texture atlases by aggregating the information in texture space. Using our automatically generated material texture atlases we demonstrate relighting and object insertion in real scenes.Learning-based tasks require large amounts of data with variety to learn the task efficiently. Using synthetic datasets to train is the norm but using traditional rendering to render large datasets is time consuming providing limited variability. We propose \textbf{a new neural rendering-based approach that learns a neural scene representation with variability and use it to generate large amounts of data at a significantly faster rate on the fly}. We demonstrate the advantage of using neural rendering as compared to traditional rendering in terms of speed of generating dataset as well as learning auxiliary tasks given the same computational budget.; L’informatique graphique a pour but de rendre des images de synthèse semblables à des photographies. Plusieurs algorithmes de rendu ont été développés au cours du dernier demi-siècle, principalement pour restituer des scènes à base d'éléments 3D créés par des artistes. Alors que les scènes initiales étaient assez simples, des représentations plus complexes de la géométrie, des matériaux et de l'éclairage ont été développés. Créer des scènes aussi complexes nécessite beaucoup de travail et de compétences de la part d'artistes 3D professionnels. Au même temps, les algorithmes de rendu impliquent des techniques de simulation complexes coûteuses en temps, pour résoudre le transport global de la lumière dans une scène.Avec la popularité grandissante de la photo numérique, le rendu basé image (IBR) a émergé comme une alternative au rendu traditionnel. Avec cette approche, l'utilisation de photos comme données d'entrée est devenue beaucoup plus rapide que la génération de scènes classiques. Les algorithmes IBR se sont d’abord concentrés sur la restitution de scènes pour en permettre une exploration libre. Au fil du temps, les modèles de scène sont devenus plus complexes et l'utilisation d'un proxy géométrique inféré à partir d’images est devenue la norme. Aujourd'hui, l'utilisation d'un maillage reconstruit à l'aide des techniques Structure-from-Motion (SfM) et Multi-view Stereo (MVS) est courante en IBR, bien que cette utilisation introduit des artefacts importants. Nous proposons d'abord un nouvel algorithme de rendu basé image, qui se concentre sur le rendu de qualité et en temps interactif d'une scène capturée}. Nous étudions différentes faiblesses des travaux précédents et proposons un algorithme qui s'appuie sur ces travaux pour obtenir de meilleurs résultats. Notre algorithme se base sur l'apparence de la surface pour traiter les régions dont l'apparence dépend de l'angle de vue différemment des régions diffuses. Hybrid-IBR obtient des résultats favorables par rapport aux approches concurrentes pour une grande variété de scènes en termes de qualité et/ou de vitesse.Bien que l'IBR soit une bonne solution de rendu, l'édition de celle-ci est difficile sans une décomposition en différents éléments : la géométrie, l'apparence des matériaux et l'éclairage de la scène. Pour notre deuxième contribution, \textbf{nous estimons explicitement les paramètres de matériaux à \emph{l'échelle de la scène} à partir d'un ensemble de photographies, pour permettre l'édition de la scène}. Alors que les solutions de photogrammétrie commerciales calculent la texture diffuse pour assister la création manuelle de matériaux, nous visons à créer \emph{automatiquement} des atlas de texture de matériaux à partir d'un ensemble d'images d'une scène. Nous nous appuyons sur les informations fournis par ces images et les transmettons à un réseau neuronal convolutif pour obtenir des cartes de matériaux pour chaque vue. En utilisant toutes ces prédictions, nous créons des atlas de texture de matériau cohérents pour toutes les vues en agrégeant les informations dans l'espace texture. Nous démontrons l'utilisation de notre atlas de texture de matériaux généré automatiquement pour rendre des scènes réelles avec un changement d’illumination et avec des objets virtuels insérés.L'apprentissage profond nécessite de grandes quantités de données variées. L'utilisation de données synthétiques est courante, mais l'utilisation du rendu traditionnel pour créer ces données prend du temps et offre une variabilité limitée. Nous proposons \textbf{une nouvelle approche basée sur le rendu neuronal qui apprend une représentation de scène neuronale avec paramètres variables, et l'utilise pour générer au vol de grandes quantités de données à un rythme beaucoup plus rapide}. Nous démontrons l'avantage d'utiliser le rendu neuronal par rapport au rendu traditionnel en termes de budget de temps, ainsi que pour l'apprentissage de tâches auxiliaires avec le même budget de calcul.

We study industrial concept sketches which are used by designers to depict and communicate the 3D shape of a product. Concept sketches are often used as a reference for 3D modeling, which is a time-consuming, manual task. Our goal is to help designers with this task by developing computational methods for reconstructing and synthesizing concept sketches. Reconstructing concept sketches into 3D is challenging because they are heavily oversketched, drawn swiftly and come without any additional annotations. Pairwise geometric relationships between 3D lines, like intersections and reflective symmetry, are unknwon beforehand, yet they are essential for reconstruction. Choosing the best 3D line relationships automatically is a difficult binary selection problem. Synthesizing concept sketches is challenging because we need to generate lines that allow designers to construct freehand sketches accurately in perspective and we need to select lines which keep the sketch readable. Improving readability while maintaining constructability via line selection also confronts us with a binary selection problem. We overcome these challenges by leveraging design principles to guide the selection of binary variables. Mathematically formulating observations about how designers construct sketches, we define constraints and objective functions to select sketch entities which best match design principles. We develop two methods for the 3D reconstruction of concept sketches and one method for the synthesis of concept sketches from CAD sequences.; Nous étudions le croquis de design industriel dont se servent les designers pour illustrer et communiquer la forme 3D d’un produit. Ces croquis sont souvent utilisés comme référence pour la modélisation en 3D, tâche chronophage et encore trop peu automatisée. Notre objectif est d’aider les designers dans cette tâche en développant des méthodes numériques pour la reconstruction et la synthèse de croquis de design. La reconstruction de croquis de design en 3D représente un défi car ces dessins sont souvent réalisés rapidement et approximativement et sans présenter d’annotations supplémentaires. Nous ne connaissons pas d’avance les relations géométriques entre les lignes 3D, comme les intersections et les correspondances de symétrie, relations binaires qui sont pourtant essentielles pour la reconstruction. Choisir les meilleures relations binaires de manière automatique est un problème combinatoire difficile. La synthèse de croquis de design représente un défi car nous devons générer des lignes permettant au designer de construire le dessin à main levée et nous devons sélectionner des lignes qui assurent la lisibilité du dessin. Améliorer la lisibilité du dessin tout en assurant sa constructibilité nous mettent encore une fois face à un problème combinatoire en variables binaires. Nous surmontons ces problèmes en étudiant des principes de design pour guider la sélection des variables binaires. Nos observations sur le processus de construction de dessin nous permettent de formuler mathématiquement des contraintes et des fonctions objectif pour sélectionner les éléments de dessin qui correspondent le mieux à nos principes de design. Nous développons deux méthodes pour la reconstruction de croquis de design en 3D et une méthode pour la synthèse de croquis de design à partir d’une séquence CAD.

International audience; Printing-on-fabric is an affordable and practical method for creating self-actuated deployable surfaces: thin strips of plastic are depositedon top of a pre-stretched piece of fabric using a commodity 3D printer; the structure, once released, morphs to a programmed 3Dshape. Several physics-aware modeling tools have recently been proposed to help designing such surfaces. However, existing simulators do not capture well all the deformations these structures can exhibit. In this work, we propose a new model for simulating printed-on-fabric composites based on a tailored bilayer formulation for modeling plastic-on-top-of-fabric strips, and an extended Saint-Venant–Kirchhoff material law for modeling the surrounding stretchy fabric. We show how to calibrate our model through a series of standard experiments. Finally, we demonstrate the improvedaccuracy of our simulator by conducting various tests.

We introduce a neural relighting algorithm for captured indoors scenes, that allows interactive free-viewpoint navigation. Our method allows illumination to be changed synthetically, while coherently rendering cast shadows and complex glossy materials. We start with multiple images of the scene and a three-dimensional mesh obtained by multi-view stereo (MVS) reconstruction. We assume that lighting is well explained as the sum of a view-independent diffuse component and a view-dependent glossy term concentrated around the mirror reflection direction. We design a convolutional network around input feature maps that facilitate learning of an implicit representation of scene materials and illumination, enabling both relighting and free-viewpoint navigation. We generate these input maps by exploiting the best elements of both image-based and physically based rendering. We sample the input views to estimate diffuse scene irradiance, and compute the new illumination caused by user-specified light sources using path tracing. To facilitate the network's understanding of materials and synthesize plausible glossy reflections, we reproject the views and compute mirror images . We train the network on a synthetic dataset where each scene is also reconstructed with MVS. We show results of our algorithm relighting real indoor scenes and performing free-viewpoint navigation with complex and realistic glossy reflections, which so far remained out of reach for view-synthesis techniques.

International audience; We present a novel method for authoring landscapes with flora and fauna while considering their mutual interactions. Our algorithm outputs a steadystate ecosystem in the form of density maps for each species, their daily circuits, and a modified terrain with eroded trails from a terrain, climatic conditions, and species with related biological information. We introduce the Resource Access Graph, a new data structure that encodes both interactions between food chain levels and animals traveling between resources over the terrain. A novel competition algorithm operating on this data progressively computes a steady-state solution up the food chain, from plants to carnivores. The user can explore the resulting landscape, where plants and animals are instantiated on the fly, and interactively edit it by over-painting the maps. Our results show that our system enables the authoring of consistent landscapes where the impact of wildlife is visible through animated animals, clearings in the vegetation, and eroded trails. We provide quantitative validation with existing ecosystems and a user-study with expert paleontologist end-users, showing that our system enables them to author and compare different ecosystems illustrating climate changes over the same terrain while enabling relevant visual immersion into consistent landscapes.

International audience; Concept sketches, ubiquitously used in industrial design, are inherently imprecise yet highly effective at communicating 3D shape to human observers. We present a new symmetry-driven algorithm for recovering designer-intended 3D geometry from concept sketches. We observe that most concept sketches of human-made shapes are structured around locally symmetric building blocks, defined by triplets of orthogonal symmetry planes. We identify potential building blocks using a combination of 2D symmetries and drawing order. We reconstruct each such building block by leveraging a combination of perceptual cues and observations about designer drawing choices. We cast this reconstruction as an integer programming problem where we seek to identify, among the large set of candidate symmetry correspondences formed by approximate pen strokes, the subset that results in the most symmetric and well-connected shape. We demonstrate the robustness of our approach by reconstructing 82 sketches, which exhibit significant over-sketching, inaccurate perspective, partial symmetry, and other imperfections. In a comparative study, participants judged our results as superior to the state-of-the-art by a ratio of 2:1.

International audience; We propose a method to transform unstructured 3D sketches into piecewise smooth surfaces that preserve sketched geometric features. Immersive 3D drawing and sketch-based 3D modeling applications increasingly produce imperfect and unstructured collections of 3D strokes as design output. These 3D sketches are readily perceived as piecewise smooth surfaces by viewers, but are poorly handled by existing 3D surface techniques tailored to wellconnected curve networks or sparse point sets. Our algorithm is aligned with human tendency to imagine the strokes as a small set of simple smooth surfaces joined along stroke boundaries. Starting with an initial proxy surface, we iteratively segment the surface into smooth patches joined sharply along some strokes, and optimize these patches to fit surrounding strokes. Our evaluation is fourfold: we demonstrate the impact of various algorithmic parameters, we evaluate our method on synthetic sketches with known ground truth surfaces, we compare to prior art, and we show compelling results on more than 50 designs from a diverse set of 3D sketch sources.

Creating realistic images of virtual scenes is a process that involves simulating light interactions, traditionally through path tracing and Monte Carlo methods, as light gets transmitted and reflected before reaching the virtual camera. This process is being used extensively in different industries such as movies, video games, physical simulations and architectural design. Monte Carlo methods, in a path tracing context, can handle complex lighting effects but the resulting images are plagued with noise which is reduced by simulating additional paths. This can be computationally expensive and a lot of research has gone into making it more efficient and accurate. While methods in the past have focused on improving the sampling quality of path tracing, recently neural networks have gained popularity as a way to render synthetic or captured scenes. This shift towards a neural augmented rendering pipeline is reflected in the methods proposed in this thesis where increasingly more aspects of rendering are handled by neural networks. A key component in this task is the scene representation of choice with many alternatives being proposed. We demonstrate how radiance fields is a good fit as they can be used to reduce noise in traditional path tracing and also be learned efficiently by neural networks. First we propose a method to inject our knowledge of the scene materials in an approximation of radiance fields to improve sampling, especially in scenes with glossy materials. Next we show that when training a network to represent radiance fields for variable scenes uniform sampling of the scene configurations leads to poor results. Instead we actively explore the space of possible scene configurations and use the network to interactively render variable scenes with hard effects, such as caustics. Even though we use a network for the final rendering our explicit scene representation vector preserves the artistic control over the scene's objects, materials and emitters. Finally we develop a generative model for mesoscale materials with complex structure and appearance. Here we use volumetric radiance fields and we condition our network on geometry and appearance parameters for artistic control of the materials represented, which is crucial in our context.; La création d'images réalistes de scènes virtuelles est un processus qui implique de simuler les interactions de la lumière, ce qui est fait traditionnellement par des méthodes de tracé de chemins et de Monte Carlo, la lumière étant transmise et réfléchie avant d'atteindre la caméra virtuelle. Ce processus est largement utilisé dans différents secteurs tels que le cinéma, les jeux vidéo, les simulations physiques et la conception architecturale. Les méthodes de Monte Carlo, dans un contexte de tracé de chemins, peuvent gérer des effets d'éclairage complexes, mais les images résultantes sont bruitées; pour réduire le bruit il faut simuler des chemins supplémentaires. Ce calcul peut être coûteux ; de nombreuses recherches ont été menées pour le rendre plus efficace et plus précis. Si, par le passé, les méthodes se sont concentrées sur l'amélioration de la qualité d'échantillonnage du tracé de chemins, les réseaux neuronaux ont récemment gagné en popularité comme moyen de rendre des scènes synthétiques ou capturées. Cette évolution vers un pipeline de rendu augmenté par les réseaux de neurones se reflète dans les méthodes proposées dans cette thèse où de plus en plus d'aspects du rendu sont traités par les réseaux de neurones. Un élément clé de cette tâche est le choix de la représentation de la scène, avec de nombreuses alternatives proposées. Nous démontrons que les champs de radiance sont une bonne solution car ils peuvent être utilisés pour réduire le bruit dans le tracé de chemins traditionnel et être appris efficacement par les réseaux neuronaux. Tout d'abord, nous proposons une méthode permettant d'injecter notre connaissance des matériaux de la scène dans une approximation des champs de radiance afin d'améliorer l'échantillonnage, en particulier dans les scènes comportant des matériaux brillants. Ensuite, nous montrons que lors de l'apprentissage d'un réseau pour représenter les champs de radiance pour des scènes variables, un échantillonnage uniforme des configurations de la scène conduit à de mauvais résultats. Au lieu de cela, nous explorons activement l'espace des configurations de scènes possibles et utilisons le réseau pour rendre de manière interactive des scènes variables avec des effets complexes, tels que les caustiques. Même si nous utilisons un réseau pour le rendu final, notre vecteur explicite de représentation de la scène préserve le contrôle artistique sur les objets, les matériaux et les émetteurs de la scène. Enfin, nous développons un modèle génératif pour les matériaux à méso-échelle avec une structure et une apparence complexes. Ici, nous utilisons des champs de radiance volumétriques et nous conditionnons notre réseau à des paramètres de géométrie et d'apparence pour un contrôle artistique des matériaux représentés, ce qui est crucial dans notre contexte.

Creative processes of artists often start with hand-drawn sketches illustrating an object. Pre-visualizing these keyframes is especially challenging when applied to volumetric materials such as smoke. The authored 3D density volumes must capture realistic flow details and turbulent structures, which is highly non-trivial and remains a manual and time-consuming process. We therefore present a method to compute a 3D smoke density field directly from 2D artist sketches, bridging the gap between early-stage prototyping of smoke keyframes and pre-visualization. From the sketch inputs, we compute an initial volume estimate and optimize the density iteratively with an updater CNN. Our differentiable sketcher is embedded into the end-to-end training, which results in robust reconstructions. Our training data set and sketch augmentation strategy are designed such that it enables general applicability. We evaluate the method on synthetic inputs and sketches from artists depicting both realistic smoke volumes and highly non-physical smoke shapes. The high computational performance and robustness of our method at test time allows interactive authoring sessions of volumetric density fields for rapid prototyping of ideas by novice users. ISSN:1467-8659 ISSN:0167-7055

We present a method to transfer the appearance of one or a few exemplar SVBRDFs to a target image representing similar materials. Our solution is extremely simple: we fine-tune a deep appearance-capture network on the provided exemplars, such that it learns to extract similar SVBRDF values from the target image. We introduce two novel material capture and design workflows that demonstrate the strength of this simple approach. Our first workflow allows to produce plausible SVBRDFs of large-scale objects from only a few pictures. Specifically, users only need take a single picture of a large surface and a few close-up flash pictures of some of its details. We use existing methods to extract SVBRDF parameters from the close-ups, and our method to transfer these parameters to the entire surface, enabling the lightweight capture of surfaces several meters wide such as murals, floors and furniture. In our second workflow, we provide a powerful way for users to create large SVBRDFs from internet pictures by transferring the appearance of existing, pre-designed SVBRDFs. By selecting different exemplars, users can control the materials assigned to the target image, greatly enhancing the creative possibilities offered by deep appearance capture., Published in Computer Graphics Forum, 39(4); Proceedings of the Eurographics Symposium on Rendering 2020

Printing-on-fabric is a recent manufacturing technique which consists in extruding molten plastic into pre-stretched fabric using standard additive manufacturing technologies. Printing-on-fabric recently gained popularity as a low-cost fabrication technique for designing deployable structures which can pop out of shape when the fabric is released. The complex morphing behavior of this composite material can be explained by metric frustration phenomena which are described in this thesis. We leverage these phenomena to build several applications aiding the design of self-actuated, lightweight structures.We first present a (forward) design tool for fabricating lightweight architectural models based on a tiling of star patterns, with the dimensions (and hence physical properties) of the individual pattern elements varying over space. Users of this system design free-form shapes by adjusting the star pattern; our system then automatically simulates the complex physical coupling between the fabric and stars to translate the design edits into shape variations.We then extend the form-finding tool to a general-purpose simulation method that can predict the buckling behavior of a variety of printed-on-fabric designs besides star tilings. We show the importance of modelling the bilayer effect caused by the differential compression at the interface between the plastic and fabric layers, as well as the anisotropic properties of the fabric.Finally, an inverse design tool that is able to reproduce target shapes with printing-on-fabric using a dense pattern of closely-spaced ribbons is introduced. The core of this method is a parameterization algorithm that bounds surface distortions along and across principal curvature directions, along with a pattern synthesis algorithm that covers a surface with ribbons to match the target distortions and curvature given by the aforementioned parameterization.; L'impression sur tissu est une technique de fabrication récente consistant à déposer sur un tissu étiré du plastique en fusion en utilisant des procédés standard de fabrication additive. L'impression sur tissu a récemment gagné en popularité car c'est une technique de fabrication peu coûteuse permettant de créer des structures déployables qui prennent automatiquement une forme en 3D lorsque le tissu est relâché. Le comportement en déformation de ce matériau composite peut s'expliquer par des phénomènes de frustration de métrique que nous décrivons dans la suite de cet exposé. Nous exploitons ces phénomènes pour proposer plusieurs applications aidant à la conception de structures auto-déployables légères.La première application présentée est un outil de design pour fabriquer des maquettes architecturales légères basé sur un pavage de motifs en étoile dans lequel les dimensions (et donc les propriétés physiques) de chaque élément varient dans l'espace. Les utilisateurs de ce système peuvent créer des formes libres en ajustant le motif d'étoiles ; notre système simule ensuite le couplage entre le tissu et les motifs de plastique pour traduire les paramètres en design en variations de forme.L'outil de simulation introduit précédemment est ensuite généralisé pour pouvoir s'appliquer à tous types de motifs imprimés et pas seulement des motifs en étoile. Nous modélisons notamment l'anisotropie du tissu ainsi que l'effet bilame causé par le différentiel de compression à l'interface entre la couche de plastique et la couche de tissu.Pour finir, une méthode de design inverse capable de reproduire des formes cibles en imprimant un motif dense de rubans peu espacés est présentée. Le cœur de la méthode consiste en un algorithme de paramétrisation qui borne les distorsions locales de surface le long des directions principales de courbure, ainsi qu'un algorithme de synthèse de motifs qui couvre la surface de rubans pour respecter les distorsions cibles et la courbure données par la paramétrisation.

This paper introduces the Instructed Glacier Model (IGM) – a model that simulates ice dynamics, mass balance and its coupling to predict the evolution of glaciers, icefields or ice sheets. The nov elty of IGM is that it models the ice flow by a Convolutional Neural Network, which is trained from data generated with hybrid SIA + SSA or Stokes ice flow models. By doing so, the most com putationally demanding model component is substituted by a cheap emulator. Once trained with representative data, we demonstrate that IGM permits to model mountain glaciers up to 1000 × faster than Stokes ones on Central Processing Units (CPU) with fidelity levels above 90% in terms of ice flow solutions leading to nearly identical transient thickness evolution. Switching to the GPU often permits additional significant speed-ups, especially when emulating Stokes dynamics or/and modelling at high spatial resolution. IGM is an open-source Python code which deals with two-dimensional (2-D) gridded input and output data. Together with a companion library of trained ice flow emulators, IGM permits user-friendly, highly efficient and mechanically state of-the-art glacier and icefields simulations, Journal of Glaciology, 68 (270), ISSN:0022-1430, ISSN:1727-5652

Neural rendering algorithms introduce a fundamentally new approach for photorealistic rendering, typically by learning a neural representation of illumination on large numbers of ground truth images. When training for a given variable scene, i.e., changing objects, materials, lights and viewpoint, the space D of possible training data instances quickly becomes unmanageable as the dimensions of variable parameters increase. We introduce a novel Active Exploration method using Markov Chain Monte Carlo, which explores D, generating samples (i.e., ground truth renderings) that best help training and interleaves training and on-the-fly sample data generation. We introduce a self-tuning sample reuse strategy to minimize the expensive step of rendering training samples. We apply our approach on a neural generator that learns to render novel scene instances given an explicit parameterization of the scene configuration. Our results show that Active Exploration trains our network much more efficiently than uniformly sampling, and together with our resolution enhancement approach, achieves better quality than uniform sampling at convergence. Our method allows interactive rendering of hard light transport paths (e.g., complex caustics) -- that require very high samples counts to be captured -- and provides dynamic scene navigation and manipulation, after training for 5-18 hours depending on required quality and variations., Comment: Project page: http://repo-sam.inria.fr/fungraph/active-exploration/

There has recently been great interest in neural rendering methods. Some approaches use 3D geometry reconstructed with Multi-View Stereo (MVS) but cannot recover from the errors of this process, while others directly learn a volumetric neural representation, but suffer from expensive training and inference. We introduce a general approach that is initialized with MVS, but allows further optimization of scene properties in the space of input views, including depth and reprojected features, resulting in improved novel-view synthesis. A key element of our approach is our new differentiable point-based pipeline, based on bi-directional Elliptical Weighted Average splatting, a probabilistic depth test and effective camera selection. We use these elements together in our neural renderer, that outperforms all previous methods both in quality and speed in almost all scenes we tested. Our pipeline can be applied to multi-view harmonization and stylization in addition to novel-view synthesis., https://repo-sam.inria.fr/fungraph/differentiable-multi-view/

Fluid control often uses optimization of control forces that are added to a simulation at each time step, such that the final animation matches a single or multiple target density keyframes provided by an artist. The optimization problem is strongly under-constrained with a high-dimensional parameter space, and finding optimal solutions is challenging, especially for higher resolution simulations. In this paper, we propose two novel ideas that jointly tackle the lack of constraints and high dimensionality of the parameter space. We first consider the fact that optimized forces are allowed to have divergent modes during the optimization process. These divergent modes are not entirely projected out by the pressure solver step, manifesting as unphysical smoke sources that are explored by the optimizer to match a desired target. Thus, we reduce the space of the possible forces to the family of strictly divergence-free velocity fields, by optimizing directly for a vector potential. We synergistically combine this with a smoothness regularization based on a spectral decomposition of control force fields. Our method enforces lower frequencies of the force fields to be optimized first by filtering force frequencies in the Fourier domain. The mask-growing strategy is inspired by Kolmogorov's theory about scales of turbulence. We demonstrate improved results for 2D and 3D fluid control especially in higher-resolution settings, while eliminating the need for manual parameter tuning. We showcase various applications of our method, where the user effectively creates or edits smoke simulations. ISSN:1467-8659 ISSN:0167-7055

International audience; We describe a new meta-material for fabricating lightweight architectural models, consisting of a tiled plastic star pattern layered over pre-stretched fabric, and an interactive system for computer-aided design of doubly-curved forms using this meta-material. 3D-printing plastic rods over pre-stretched fabric recently gained popularity as a low-cost fabrication technique for complex free-form shapes that automatically lift in space. Our key insight is to focus on rods arranged into repeating star patterns, with the dimensions (and hence physical properties) of the individual pattern elements varying over space. Our star-based meta-material on the one hand allows effective form-finding due to its low-dimensional design space, while on the other is flexible and powerful enough to express large-scale curvature variations. Users of our system design free-form shapes by adjusting the star pattern; our system then automatically simulates the complex physical coupling between the fabric and stars to translate the design edits into shape variations. We experimentally validate our system and demonstrate strong agreement between the simulated results and the final fabricated prototypes.

International audience; Image-based rendering (IBR) provides a rich toolset for free-viewpoint navigation in captured scenes. Many methods exist, usually with an emphasis either on image quality or rendering speed. In this paper we identify common IBR artifacts and combine the strengths of different algorithms to strike a good balance in the speed/quality tradeoff. First, we address the problem of visible color seams that arise from blending casually-captured input images by explicitly treating view-dependent effects. Second, we compensate for geometric reconstruction errors by refining per-view information using a novel clustering and filtering approach. Finally, we devise a practical hybrid IBR algorithm, which locally identifies and utilizes the rendering method best suited for an image region while retaining interactive rates. We compare our method against classical and modern (neural) approaches in indoor and outdoor scenes and demonstrate superiority in quality and/or speed.

International audience; We present CASSIE, a conceptual modeling system in VR that leverages freehand mid-air sketching, and a novel 3D optimization framework to create connected curve network armatures, predictively surfaced using patches with C0 continuity. Our system provides a judicious balance of interactivity and automation, providing a homogeneous 3D drawing interface for a mix of freehand curves, curve networks, and surface patches. Our system encourages and aids users in drawing consistent networks of curves, easing the transition from freehand ideation to concept modeling. A comprehensive user study with professional designers as well as amateurs (N=12), and a diverse gallery of 3D models, show our armature and patch functionality to offer a user experience and expressivity on par with freehand ideation, while creating sophisticated concept models for downstream applications.

International audience; Fashion design often starts with hand-drawn, expressive sketches that communicate the essence of a garment over idealized human bodies. We propose an approach to automatically dress virtual characters from such input, previously complemented with user-annotations. In contrast to prior work requiring users to draw garments with accurate proportions over each virtual character to be dressed, our method follows a style transfer strategy : the information extracted from a single, annotated fashion sketch can be used to inform the synthesis of one to many new garment(s) with similar style, yet different proportions. In particular, we define the style of a loose garment from its silhouette and folds, which we extract from the drawing. Key to our method is our strategy to extract both shape and repetitive patterns of folds from the 2D input. As our results show, each input sketch can be used to dress a variety of characters of different morphologies, from virtual humans to cartoon-style characters.

International audience; Image-Based Rendering allows users to easily capture a scene using a single camera and then navigate freely with realistic results. However, the resulting renderings are completely static, and dynamic effects – such as fire, waterfalls or small waves – cannot be reproduced. We tackle the challenging problem of enabling free-viewpoint navigation including such stationary dynamic effects, but still maintaining the simplicity of casual capture. Using a single camera – instead of previous complex synchronized multi-camera setups – means that we have unsynchronized videos of the dynamic effect from multiple views, making it hard to blend them when synthesizing novel views. We present a solution that allows smooth free-viewpoint video-based rendering (VBR) of such scenes using temporal Laplacian pyramid decomposition video, enabling spatio-temporal blending. For effects such as fire and waterfalls, that are semi-transparent and occupy 3D space, we first estimate their spatial volume. This allows us to create per-video geometries and alpha-matte videos that we can blend using our frequency-dependent method. We also extend Laplacian blending to the temporal dimension to remove additional temporal seams. We show results on scenes containing fire, waterfalls or rippling waves at the seaside, bringing these scenes to life.

International audience; Recent rendering advances dramatically reduce the cost of global illumination. But even with hardware acceleration, complex light paths with multiple glossy interactions are still expensive; our new algorithm stores these paths in precomputed light probes and reprojects them at runtime to provide interactivity. Combined with traditional light maps for diffuse lighting our approach interactively renders all light paths in static scenes with opaque objects. Naively reprojecting probes with glossy lighting is memory-intensive, requires efficient access to the correctly reflected radiance, and exhibits problems at occlusion boundaries in glossy reflections. Our solution addresses all these issues. To minimize memory, we introduce an adaptive light probe parameterization that allocates increased resolution for shinier surfaces and regions of higher geometric complexity. To efficiently sample glossy paths, our novel gathering algorithm reprojects probe texels in a view-dependent manner using efficient reflection estimation and a fast rasterization-based search. Naive probe reprojection often sharpens glossy reflections at occlusion boundaries, due to changes in parallax. To avoid this, we split the convolution induced by the BRDF into two steps: we precompute probes using a lower material roughness and apply an adaptive bilateral filter at runtime to reproduce the original surface roughness. Combining these elements, our algorithm interactively renders complex scenes while fitting in the memory, bandwidth, and computation constraints of current hardware.

Computer-generated imagery (CGI) takes a growing place in our everyday environment. Whether it is in video games or movies, CGI techniques are constantly improving in quality but also require ever more qualitative artistic content which takes a growing time to create. With the emergence of virtual and augmented reality, often comes the need to render or re-render assets that exist in our world. To allow widespread use of CGI in applications such as telepresence or virtual visits, the need for manual artistic replication of assets must be removed from the process. This can be done with the help of Image-Based Rendering (IBR) techniques that allow scenes or objects to be rendered in a free-viewpoint manner from a set of sparse input photographs. While this process requires little to no artistic work, it also does not allow for artistic control or editing of scene content.In this dissertation, we explore Multi-view Image Editing and Rendering. To allow casually captured scenes to be rendered with content alterations such as object removal, lighting edition, or scene compositing, we leverage the use of optimization techniques and modern deep-learning. We design our methods to take advantage of all the information present in multi-view content while handling specific constraints such as multi-view coherency.For object removal, we introduce a new plane-based multi-view inpainting algorithm. Planes are a simple yet effective way to fill geometry and they naturally enforce multi-view coherency as inpainting is computed in a shared rectified texture space, allowing us to correctly respect perspective. We demonstrate instance-based object removal at the scale of a street in scenes composed of several hundreds of images.We next address outdoor relighting with a learning-based algorithm that efficiently allows the illumination in a scene to be changed, while removing and synthesizing cast shadows for any given sun position and accounting for global illumination. An approximate geometric proxy built using multi-view stereo is used to generate illumination and shadow related image buffers that guide a neural network. We train this network on a set of synthetic scenes allowing full supervision of the learning pipeline. Careful data augmentation allows our network to transfer to real scenes and provides state of the art relighting results. We also demonstrate the capacity of this network to be used to compose real scenes captured under different lighting conditions and orientation.We then present contributions to image-based rendering quality. We discuss how our carefully designed depth-map meshing and simplification algorithm improve rendering performance and quality of a new learning-based IBR method.Finally, we present a method that combines relighting, IBR, and material analysis. To enable relightable IBR with accurate glossy effects, we extract both material appearance variations and qualitative texture information from multi-view content in the form of several IBR heuristics. We further combine them with path-traced irradiance images that specify the input and target lighting. This combination allows a neural network to be trained to implicitly extract material properties and produce realistic-looking relit viewpoints. Separating diffuse and specular supervision is crucial in obtaining high-quality output., Les images de synthèse (CGI) prennent une place grandissante dans notre environnement. Que ce soit dans les jeux vidéos ou les films, leur qualité ne cesse de s’accroître nécessitant la création fastidieuse de contenus artistiques. L’émergence de la réalité virtuelle et augmentée, entraine la nécessité de rendre des environnements existants. Pour permettre l’utilisation généralisée des images de synthèse dans des applications telles que la télé-présence ou les visites virtuelles, la digitalisation manuelle des contenus par des artistes se doit d’être évitée. Une des solutions peut provenir des techniques de Rendu à Base d’Images (IBR) qui permettent de rendre des scènes, depuis un point de vue libre, à partir d’un ensemble de photographies parcimonieux. Bien que ces méthodes ne nécessitent que peu de travail artistique, elles n’autorisent cependant pas le contrôle ou l’édition du contenu.Dans cette thèse, nous explorons l’Edition et le Rendu d’Images Multi-vues. Afin de permettre à des scènes, capturées avec le moins de contraintes possibles, d’être rendues avec des altérations telles que la suppression d’objets, l’édition d’éclairage, ou la composition de scènes, nous exploitons les techniques d’optimisation et d’apprentissage profond. Nous concevons nos méthodes afin qu’elles tirent pleinement avantage de l’information présente dans le contenu multi-vues, tout en en respectant ses contraintes spécifiques.Pour la suppression d’objets, nous introduisons un algorithme de remplissage automatique, multi-vues cohérent, utilisant une représentation planaire. Les plans sont des objets simples et efficaces pour combler la géométrie, dont la cohérence multi-vues émerge naturellement lorsque le remplissage est effectué dans un espace texture rectifié et partagé. Ils permettent aussi le respect des effets de perspective. Nous démontrons la capacité d’enlever des objets, à grande l’échelle, dans des scènes contenant plusieurs centaines d’images.Nous traitons ensuite le problème du rééclairage des scènes extérieures par une méthode d’apprentissage profond. Elle permet de modifier l’illumination, en enlevant et synthétisant les ombres portées, pour une position du soleil quelconque, tout en tenant compte des variations d’illumination globale. Une représentation géométrique approximative, reconstruite en utilisant la stéréo multi-vues, est utilisée pour générer des images tampons d’illumination et d’ombres qui guident un réseau de neurones. Nous entrainons ce réseau sur un ensemble de scènes synthétiques, permettant une supervision complète. Une augmentation des données minutieuse permet à notre réseau de généraliser aux scènes réelles et de produire l’état de l’art en terme de résultats. Nous démontrons ensuite, la capacité du réseau à être utilisé pour composer des scènes réelles, capturées dans des conditions d’orientation et d’éclairages différentes. Nous présentons ensuite des contributions à la qualité de l'IBR. Nous introduisons un algorithme de maillage de cartes de profondeur et de leur simplification. Nous démontrons son impact sur la qualité et les performances d’une nouvelle méthode d’IBR utilisant l’apprentissage.Enfin, nous introduisons une méthode qui combine rééclairage, IBR, et analyse de matériaux. Afin de permettre un rendu à base d’images, rééclairable et tenant compte des effets spéculaires, nous extrayons du contenu multi-vues les variations d’apparence des matériaux et l’information de texture haute résolution, sous la forme de plusieurs rendus IBR heuristiques. Nous les combinons ensuite avec des rendus d’irradiance, obtenus par lancer de rayons, qui spécifient les conditions d’éclairage initiales et désirées. Cette combinaison permet d’entrainer un réseau de neurones à extraire implicitement les propriétés des matériaux et à produire des points de vue rééclairés réalistes [...]

We present a sketch-based CAD modeling system, where users create objects incrementally by sketching the desired shape edits, which our system automatically translates to CAD operations. Our approach is motivated by the close similarities between the steps industrial designers follow to draw 3D shapes, and the operations CAD modeling systems offer to create similar shapes. To overcome the strong ambiguity with parsing 2D sketches, we observe that in a sketching sequence, each step makes sense and can be interpreted in the context of what has been drawn before. In our system, this context corresponds to a partial CAD model, inferred in the previous steps, which we feed along with the input sketch to a deep neural network in charge of interpreting how the model should be modified by that sketch. Our deep network architecture then recognizes the intended CAD operation and segments the sketch accordingly, such that a subsequent optimization estimates the parameters of the operation that best fit the segmented sketch strokes. Since there exists no datasets of paired sketching and CAD modeling sequences, we train our system by generating synthetic sequences of CAD operations that we render as line drawings. We present a proof of concept realization of our algorithm supporting four frequently used CAD operations. Using our system, participants are able to quickly model a large and diverse set of objects, demonstrating Sketch2CAD to be an alternate way of interacting with current CAD modeling systems.

The creation and interactive rendering of realistic environments is a time consuming problem requiring human interaction and tweaking at all steps. Image-based approaches use viewpoints of a real world or high quality synthetic scene to simplify these tasks. These viewpoints can be captured in the real world or generated with accurate offline techniques. When rendering a novel viewpoint, geometry and lighting information are inferred from the data in existing views, allowing for interactive exploration of the environment. But sparse coverage of the scene in the spatial or angular domain introduces artifacts; a small number of input images is adversarial to real world scene reconstruction, as is the presence of complex materials with view-dependent effects when rendering both real and synthetic scenes. We aim at lifting these constraints by refining the way information is stored and aggregated from the viewpoints and exploiting redundancy. We also design geometric tools to properly reproject view-dependent effects in the novel view.For real world scenes, we rely on semantic information to infer material and object placement. We first introduce a method to perform reconstruction of architectural elements from a set of three to five photographs of a scene. We exploit the repetitive nature of such elements to extract them and aggregate their color and geometry information, generating a common ideal representation that can then be reinserted in the initial scene. Aggregation helps improve the accuracy of the viewpoint pose estimation, of the element geometry reconstruction, and is used to detect locations exhibiting view-dependent specular effects.We describe a second method designed to similarly rely on semantic scene information to improve rendering of street-level scenery. In these scenes cars exhibit many view-dependent effects that make them hard to reconstruct and render accurately. We detect and extract them, relying on a per-object view parameterization to refine their geometry, including a simplified representation of reflective surfaces such as windows. To render view-dependent effects we perform a specular layer separation and use our analytical reflector representation to reproject the information in the novel view following the flow of reflections.Finally, while synthetic scenes provide completely accurate geometric and material information, rendering high quality view-dependent effects interactively is still difficult. Inspired by our second method, we propose a novel approach to render such effects using precomputed global illumination. We can precompute specular information at a set of predefined locations in the scene, stored in panoramic probes. Using a simplified geometric representation we robustly estimate the specular flow, gathering information from the probes at a novel viewpoint. Combined with an adaptive parameterization of the probes and a material-aware reconstruction filter, we render specular and glossy effects interactively.The results obtained with our methods show an improvement in the quality of recreated view-dependent effects, using both synthetic and real world data and pave the way for further research in image-based techniques for realistic rendering., La création et le rendu interactif d’environnements réalistes est un problème complexe qui requiert de nombreux réglages et ajustements manuels. Les méthodes basées-image visent à simplifier ces tâches en utilisant des vues existantes d’une scène réelle ou synthétique. Ces points de vue peuvent être capturés dans le monde réel ou générés grâce à des algorithmes de rendu hors-ligne très réalistes. Pour générer un nouveau point de vue sur la scène, la géométrie et l’apparence associées sont estimées à partir de l’information des vues existantes, permettant l’exploration de l’environnement en temps réel.Lorsque les vues d’entrée ne couvrent la scène que partiellement, spatialement ou angulairement, des artefacts apparaissent souvent ; utiliser un faible nombre d’images d’entrée limite la qualité de la reconstruction des scènes réelles, tandis que les matériaux non-diffus compliquent le rendu des scènes tant réelles que synthétiques. Nous proposons plusieurs méthodes pour contourner ces limitations, en améliorant la façon dont l’information est stockée et agrégée depuis les vues d’entrée tout en exploitant les redondances. Nous décrivons aussi des outils géométriques afin de reprojeter les effets dépendants du point de vue pour la génération d’une nouvelle vue.Pour les scènes tirées du monde réel, nous détectons la présence de matériaux et objets grâce à de l’information sémantique. Nous présentons tout d’abord une méthode pour reconstruire et rendre des éléments architecturaux à partir de quelques vues. Nous exploitons la nature répétitive de ces éléments, extrayant et combinant leur information d’apparence et de géométrie pour générer une représentation commune idéale qui peut par la suite être réinsérée dans la scène initiale. Cette combinaison améliore l’estimation des paramètres des vues d’entrée, la reconstruction de la géométrie de l’élément, et est utile pour détecter les régions comportant des effets spéculaires dépendants du point de vue.Nous décrivons une deuxième méthode qui s’appuie aussi sur la sémantique de la scène pour améliorer le rendu d’environnements urbains. Dans ces scènes, les voitures présentent de nombreux effets dépendants du point de vue qui les rendent complexes à reconstruire et rendre fidèlement. Nous détectons et extrayons ces éléments grâce à une paramétrisation spécifique à chaque instance, afin de raffiner leur géométrie, tout en construisant une représentation simplifiée des surfaces réflectives, telles que les fenêtres. Pour rendre les effets dépendants du point de vue, nous séparons l’information spéculaire de chaque vue d’entrée et utilisons notre représentation analytique des réflecteurs pour reprojeter l’information dans la nouvelle vue, en respectant le déplacement des réflexions.Enfin, bien que les scènes synthétiques fournissent une information exacte de la géométrie et des matériaux présents, restituer fidèlement les effets non-diffus de façon interactive reste difficile. En nous inspirant de notre deuxième méthode, nous proposons une nouvelle approche pour rendre ces effets en utilisant de l’illumination globale précalculée. Pour un ensemble de points de vue prédéfinis dans la scène, nous précalculons l’information spéculaire, stockée dans des images panoramiques. Grâce à une représentation simplifiée de la géométrie, nous pouvons estimer le déplacement des réflexions de façon robuste, accumulant dans la nouvelle vue l’information provenant des panoramas. Combinée avec une paramétrisation adaptative des images panoramiques et un filtre de reconstruction préservant les matériaux, cette méthode restitue les effets de réflexions pour différent types de matériaux, de façon interactive.Les résultats obtenus grâce à nos méthodes montrent une amélioration de la qualité du rendu des effets dépendants du point de vue, que ce soit pour des données synthétiques ou du monde réel. Ces travaux ouvrent la voie à de futures recherches sur les techniques basées-image pour le rendu réaliste.

International audience; Image-Based Rendering (IBR) has made impressive progress towards highly realistic, interactive 3D navigation for many scenes, including cityscapes. However, cars are ubiquitous in such scenes; multi-view stereo reconstruction provides proxy geometry for IBR, but has difficulty with shiny car bodies, and leaves holes in place of reflective, semi-transparent windows on cars. We present a new approach allowing free-viewpoint IBR of cars based on an approximate analytic reflection flow computation on curved windows. Our method has three components: a refinement step of reconstructed car geometry guided by semantic labels, that provides an initial approximation for missing window surfaces and a smooth completed car hull; an efficient reflection flow computation using an ellipsoid approximation of the curved car windows that runs in real-time in a shader and a reflection/background layer synthesis solution. These components allow plausible rendering of reflective, semi-transparent windows in free viewpoint navigation. We show results on several scenes casually captured with a single consumer-level camera, demonstrating plausible car renderings with significant improvement in visual quality over previous methods.

International audience; We present the first algorithm capable of automatically lifting real-world, vector-format, industrial design sketches into 3D. Targeting real-world sketches raises numerous challenges due to inaccuracies, use of overdrawn strokes, and construction lines. In particular, while construction lines convey important 3D information, they add significant clutter and introduce multiple accidental 2D intersections. Our algorithm exploits the geometric cues provided by the construction lines and lifts them to 3D by computing their intended 3D intersections and depths. Once lifted to 3D, these lines provide valuable geometric constraints that we leverage to infer the 3D shape of other artist drawn strokes. The core challenge we address is inferring the 3D connectivity of construction and other lines from their 2D projections by separating 2D intersections into 3D intersections and accidental occlusions. We efficiently address this complex combinatorial problem using a dedicated search algorithm that leverages observations about designer drawing preferences , and uses those to explore only the most likely solutions of the 3D intersection detection problem. We demonstrate that our separator outputs are of comparable quality to human annotations, and that the 3D structures we recover enable a range of design editing and visualization applications, including novel view synthesis and 3D-aware scaling of the depicted shape.

International audience; A major challenge in line drawing vectorization is segmenting the input bitmap into separate curves. This segmentation is especially problematic for rough sketches, where curves are depicted using multiple overdrawn strokes. Inspired by feature-aligned mesh quadrangulation methods in geometry processing, we propose to extract vector curve networks by parametrizing the image with local drawing-aligned integer grids. The regular structure of the grid facilitates the extraction of clean line junctions; due to the grid's discrete nature, nearby strokes are implicitly grouped together. We demonstrate that our method successfully vectorizes both clean and rough line drawings, whereas previous methods focused on only one of those drawing types.

International audience; Multi-view stereo can be used to rapidly create realistic virtual content, such as textured meshes or a geometric proxy for free-viewpoint Image-Based Rendering (IBR). These solutions greatly simplify the content creation process compared to traditional methods, but it is difficult to modify the content of the scene. We propose a novel approach to create scenes by composing (parts of) multiple captured scenes. The main difficulty of such compositions is that lighting conditions in each captured scene are different; to obtain a realistic composition we need to make lighting coherent. We propose a two-pass solution, by adapting a multi-view relighting network. We first match the lighting conditions of each scene separately and then synthesize shadows between scenes in a subsequent pass. We also improve the realism of the composition by estimating the change in ambient occlusion in contact areas between parts and compensate for the color balance of the different cameras used for capture. We illustrate our method with results on multiple compositions of outdoor scenes and show its application to multi-view image composition, IBR and textured mesh creation.

International audience; We present a method to create vector cliparts from photographs. Our approach aims at reproducing two key properties of cliparts: they should be easily editable, and they should represent image content in a clean, simplified way. We observe that vector artists satisfy both of these properties by modeling cliparts with linear color gradients, which have a small number of parameters and approximate well smooth color variations. In addition, skilled artists produce intricate yet editable artworks by stacking multiple gradients using opaque and semi-transparent layers. Motivated by these observations, our goal is to decompose a bitmap photograph into a stack of layers, each layer containing a vector path filled with a linear color gradient. We cast this problem as an optimization that jointly assigns each pixel to one or more layer and finds the gradient parameters of each layer that best reproduce the input. Since a trivial solution would consist in assigning each pixel to a different, opaque layer, we complement our objective with a simplicity term that favors decompositions made of few, semi-transparent layers. However, this formulation results in a complex combinatorial problem combining discrete unknowns (the pixel assignments) and continuous unknowns (the layer parameters). We propose a Monte Carlo Tree Search algorithm that efficiently explores this solution space by leveraging layering cues at image junctions. We demonstrate the effectiveness of our method by reverse-engineering existing cliparts and by creating original cliparts from studio photographs.

Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even human-adjusted pairs of images, we seek to reproduce the enhancements and enable real-time evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input/output images, we train a convolutional neural network to predict the coefficients of a locally-affine model in bilateral space. Our architecture learns to make local, global, and content-dependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a low-resolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edge-preserving fashion using a new slicing node, and then applies those upsampled transformations to the full-resolution image. Our algorithm processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of-the-art approximation techniques on a large class of image operators. Unlike previous work, our model is trained off-line from data and therefore does not require access to the original operator at runtime. This allows our model to learn complex, scene-dependent transformations for which no reference implementation is available, such as the photographic edits of a human retoucher., 12 pages, 14 figures, Siggraph 2017

International audience; Implementing Monte Carlo integration requires significant domain expertise. While simple samplers, such as unidirectional path tracing, are relatively forgiving, more complex algorithms, such as bidirectional path tracing or Metropolis methods, are notoriously dificult to implement correctly. We propose Aether, an embedded domain specific language for Monte Carlo integration , which offers primitives for writing concise and correct-by-construction sampling and probability code. The user is tasked with writing sampling code, while our compiler automatically generates the code necessary for evaluating PDFs as well as the book keeping and combination of multiple sampling strategies. Our language focuses on ease of implementation for rapid exploration, at the cost of run time performance. We demonstrate the effectiveness of the language by implementing several challenging rendering algorithms as well as a new algorithm, which would otherwise be prohibitively diifficult.

International audience; Texture, highlights, and shading are some of many visual cues that allow humans to perceive material appearance in pictures. Designing algorithms able to leverage these cues to recover spatially-varying bi-directional reflectance distribution functions (SVBRDFs) from a few images has challenged computer graphics researchers for decades. I explore the use of deep learning to tackle lightweight appearance capture and make sense of these visual cues. Our networks are capable of recovering per-pixel normals, diffuse albedo, specular albedo and specular roughness from as little as one picture of a flat surface lit by a hand-held flash. We propose a method which improves its prediction with the number of input pictures, and reaches high quality reconstructions with up to 10 images -- a sweet spot between existing single-image and complex multi-image approaches. We introduce several innovations on training data acquisition and network design, bringing clear improvement over the state of the art for lightweight material capture.

In stereoscopic displays, such as those used in VR/AR headsets, our eyes are presented with two different views. The disparity between the views is typically used to convey depth cues, but it could be also used to enhance image appearance. We devise a novel technique that takes advantage of binocular fusion to boost perceived local contrast and visual quality of images. Since the technique is based on fixed tone curves, it has negligible computational cost and it is well suited for real-time applications, such as VR rendering. To control the trade-off between contrast gain and binocular rivalry, we conduct a series of experiments to explain the factors that dominate rivalry perception in a dichoptic presentation where two images of different contrasts are displayed. With this new finding, we can effectively enhance contrast and control rivalry in mono- and stereoscopic images, and in VR rendering, as confirmed in validation experiments.

National audience; Deep learning achieves impressive performances on image segmentation, which has motivated the recent developmentof deep neural networks for the related task of sketch segmentation, where the goal is to assign labels to thedifferent strokes that compose a line drawing. However, while natural images are well represented as bitmaps, linedrawings can also be represented as vector graphics, such as point sequences and point clouds. In addition to offeringdifferent trade-offs on resolution and storage, vector representations often come with additional information,such as stroke ordering and speed. In this paper, we evaluate three crucial design choices for sketch segmentationusing deep-learning : which sketch representation to use, which information to encode in this representation,and which loss function to optimize. Our findings suggest that point clouds represent a competitive alternative tobitmaps for sketch segmentation, and that providing extra-geometric information improves performance.

Empowered by deep learning, recent methods for material capture can estimate a spatially-varying reflectance from a single photograph. Such lightweight capture is in stark contrast with the tens or hundreds of pictures required by traditional optimization-based approaches. However, a single image is often simply not enough to observe the rich appearance of real-world materials. We present a deep-learning method capable of estimating material appearance from a variable number of uncalibrated and unordered pictures captured with a handheld camera and flash. Thanks to an order-independent fusing layer, this architecture extracts the most useful information from each picture, while benefiting from strong priors learned from data. The method can handle both view and light direction variation without calibration. We show how our method improves its prediction with the number of input pictures, and reaches high quality reconstructions with as little as 1 to 10 images -- a sweet spot between existing single-image and complex multi-image approaches., Comment: Accepted to EGSR 2019 in the CGF track

Creating realistic images with the traditional rendering pipeline requires tedious work, starting with complex manual work to create 3D models, materials, and lighting, and then computationally expensive realistic rendering. Such a process requires both skilled artists and significant computing power. Image Based Rendering (IBR) is an alternative way to create high quality content by only using an unstructured set of photos as input. IBR allows casual users to create and render realistic and immersive scenes in real time, for applications such as virtual tourism, cultural heritage, interactive mapping, urban and architecture planning, and movie production. Existing IBR methods produce generally good image quality, but still suffer from limitations. First, many types of scene content produce visually-unappealing rendering artifacts, because the underlying scene representation is insufficient, e.g, for reflective surfaces, thin structures, and dynamic content. Second, scenes are often captured with real- world constraints which require editing to meet the user requirements, yet existing IBR methods do not allow this. To address editing, we propose to extend single image inpainting to allow sparse multiview object removal. Such inpainting requires to hallucinating both color and geometry behind the object to be removed in a multi-view coherent fashion. Our method reduces rendering artifacts by removing objects which are not well represented by IBR methods or by moving well represented objects in the scene. To address rendering quality, we enlarge the scope of casual IBR in two different ways. First we deal with the case of thin structures, which are extremely challenging for multi-view 3D reconstruction and represent a major limitation for IBR in an urban context. We propose a pipeline which locates and renders thin structures supported by simple surfaces. We introduce both a multi-view segmentation algorithm for thin structures, and a rendering method which extends traditional IBR with transparency information. Second, we propose an approach to extend IBR to dynamic contents. By focusing on time-dependent stochastic textures, we preserve both the casual capture setup and the free-viewpoint navigation of the rendered scene. Our key insight is to use a video representation which is adapted to video looping and spatio-temporal blending. Our results for all methods show improved visual quality compared to previous solutions on a variety of input scenes., La création d'images réalistes avec le processus classique de rendu demande un travail manuel considérable, de la génération de modèles 3D à la gestion de l'illumination. Cela demande à la fois des artistes experts modeleurs 3D mais également des machines avec une certaine puissance de calcul. Se basant uniquement sur des photos prises par un utilisateur lambda, le rendu basé image (IBR) est un moyen alternatif de rendre une scène en temps réel, de manière immersive et réaliste. Ce type de rendu possède des applications dans des domaines tels que le tourisme virtuel, la préservation du patrimoine, la cartographie interactive, la planification urbaine et architecturale, ainsi que la production de films. De nombreuses méthodes efficaces de rendu base image ont été proposées ces dernières années, mais elles possèdent néanmoins certaines limitations. Tout d'abord, bien que ces méthodes permettent effectivement de générer des images de bonne qualité, il est difficile de pouvoir modifier le contenu de la scène. En effet, la capture d'une scène réelle s'accompagne des contraintes liées a l'environnement au moment de la prise de photos, qui peut ne pas correspondre totalement aux exigences de l'utilisateur. Ensuite, ces méthodes dépendent grandement de la qualité de la représentation géométrique sous-jacente des scènes. En conséquence, des scènes contenant par exemple des surfaces réflectives, des structures fines ou bien du contenu dynamique, produisent des artefacts visuels important. Afin de répondre à la première limitation, nous proposons d’étendre la complétion d’image a un contexte multi-vue non structuré, permettant ainsi le retrait d’objet d’une scène. Ce genre de complétion demande non seulement d’halluciner l’apparence, mais également la géométrie de ce qui se trouve derrière l’objet à retirer. Notre méthode réduit les artefacts de rendu en supprimant les objets mal représentés par l’IBR, et permet également de déplacer des objets correctement rendus. Nous répondons à la deuxième limitation en élargissant le spectre des scènes traitable en IBR, et ce de deux manières. Tout d’abord, nous nous focalisons sur le cas des structures fines qui sont un cas particulièrement compliqué pour la reconstruction multi-vue 3D, et qui représente une importante limitation pour l’IBR dans un contexte urbain. Nous proposons une méthode qui extrait puis rend les structures fines dont la surface sous-jacente est simple. Nous introduisons un algorithme de segmentation multi-vue pour les structures fines, ainsi qu’une méthode de rendu qui étend le rendu IBR avec de l’information de transparence. Enfin, nous proposons une première approche pour étendre l’IBR à des contenus dynamiques. En nous focalisant sur des effets dynamiques stochastiques, nous sommes capables de préserver à la fois une acquisition facile à mettre en œuvre et une navigation libre dans la scène rendue. Notre idée principale est d’utiliser une représentation des vidéos adaptée à les mélanger spatio-temporellement et à les faire boucler. Les résultats de chacune de nos méthodes montrent une amélioration de la qualité visuelle de rendu sur des scènes variées.

Digital tools brings new ways of creation, for accomplished artists as well as for any individual willing to create. In this thesis, I am interested in two different aspects in helping artists: interpreting their creation and generating new content. I first study how to interpret a sketch as a 3D object. We propose a data-driven approach that tackles this challenge by training deep convolutional neural networks (CNN) to predict occupancy of a voxel grid from a line drawing. We integrate our CNNs in an interactive modeling system that allows users to seamlessly draw an object, rotate it to see its 3D reconstruction, and refine it by re-drawing from another vantage point using the 3D reconstruction as guidance. We then complement this technique with a geometric method that allows to refine the quality of the final object. To do so, we train an additional CNN to predict higher resolution normal maps from each input view. We then fuse these normal maps with the voxel grid prediction by optimizing for the final surface. We train all of these networks by rendering synthetic contour drawings from procedurally generated abstract shapes. In a second part, I present a method to generate stylized videos with a look reminiscent of traditional 2D animation. Existing stylization methods often retain the 3D motion of the original video, making the result look like a 3D scene covered in paint rather than a 2D painting of a scene. Inspired by cut-out animation, we propose to modify the motion of the sequence so that it is composed of 2D rigid motions. To achieve this goal, our approach applies motion segmentation and optimization to best approximate the input optical flow with piecewise-rigid transforms, and re-renders the video such that its content follows the simplified motion. Applying existing stylization algorithm to the new sequence produce a stylized video more similar to 2D animation. Although the two parts of my thesis lean on different methods, they both rely on traditional techniques used by artists: either by understanding how they draw objects or by taking inspiration from how they simplify the motion in 2D animation., Les outils digitaux ouvrent de nouvelles voies de création, aussi bien pour les artistes chevronnés que pour tout autre individu qui souhaite créer. Dans cette thèse, je m'intéresse à deux aspects complémentaires de ces outils : interpréter une création existante et générer du nouveau contenu. Dans une première partie, j'étudie comment interpréter un dessin comme un objet 3D. Nous proposons une approche basée donnée qui aborde cette problématique en entrainant des réseaux convolutifs profonds (CNN) à prédire l'occupation d'une grille de voxels à partir de dessins. Nous intégrons ces CNNs dans un système de modélisation interactif qui permet à l’utilisateur de dessiner un objet, tourner autour pour voir sa reconstruction 3D et le raffiner en redessinant depuis une nouvelle vue. Nous complémentons cette approche par une méthode géométrique qui permet d’améliorer la qualité de l'objet final. Pour cela, nous entrainons un CNN à prédire des cartes de normales à plus haute résolution depuis chaque vue d'entrée. Nous fusionnons alors ces cartes de normales avec la grille de voxel en optimisant pour la surface finale. Nous entrainons l'ensemble de ces réseaux grâce à des rendus de contours d'objets abstraits générés procéduralement. Dans une seconde partie, je présente une méthode pour générer des vidéos stylisées faisant penser à de l'animation traditionnelle. La plupart des méthodes existantes gardent le mouvement 3D originel de la vidéo, produisant un résultat ressemblant plus à une scène 3D couverte de peinture qu'à une peinture 2D de la scène. Inspirés par l'animation "cut-out", nous proposons de modifier le mouvement de la séquence afin qu'il soit composé de mouvements rigides en 2D. Pour y parvenir, notre approche segmente le mouvement et l'optimise afin d'approximer au mieux le flot optique d'entrée avec des transformations rigides par morceaux, et re-rend la vidéo de façon à ce que son contenu suive ce mouvement simplifié. En appliquant les méthodes de stylisations existantes sur notre nouvelle séquence, on obtient une vidéo stylisée plus proche d'une animation 2D. Ces deux parties reposent sur des méthodes différentes mais toutes deux s'appuient sur les techniques traditionnelles utilisées par les artistes : soit en comprenant comment ils dessinent un objet, soit en s'inspirant de leur façon de simplifier le mouvement.