Search

To date relatively few efforts have been made on the automatic generation of musical instrument playing animations. This problem is challenging due to the intrinsically complex, temporal relationship between music and human motion as well as the lacking of high quality music-playing motion datasets. In this paper, we propose a fully automatic, deep learning based framework to synthesize realistic upper body animations based on novel guzheng music input. Specifically, based on a recorded audiovisual motion capture dataset, we delicately design a generative adversarial network (GAN) based approach to capture the temporal relationship between the music and the human motion data. In this process, data augmentation is employed to improve the generalization of our approach to handle a variety of guzheng music inputs. Through extensive objective and subjective experiments, we show that our method can generate visually plausible guzheng-playing animations that are well synchronized with the input guzheng music, and it can significantly outperform \uline{the state-of-the-art} methods. In addition, through an ablation study, we validate the contributions of the carefully-designed modules in our framework.

Realistic 3D facial modeling and animation have been increasingly used in many graphics, animation, and virtual reality applications. However, generating realistic fine-scale wrinkles on 3D faces, in particular, on animated 3D faces, is still a challenging problem that is far away from being resolved. In this article we propose an end-to-end system to automatically augment coarse-scale 3D faces with synthesized fine-scale geometric wrinkles. By formulating the wrinkle generation problem as a supervised generation task, we implicitly model the continuous space of face wrinkles via a compact generative model, such that plausible face wrinkles can be generated through effective sampling and interpolation in the space. We also introduce a complete pipeline to transfer the synthesized wrinkles between faces with different shapes and topologies. Through many experiments, we demonstrate our method can robustly synthesize plausible fine-scale wrinkles on a variety of coarse-scale 3D faces with different shapes and expressions.

Industrial Augmented Reality (iAR) has demonstrated its advantages to communicate technical information in the fields of maintenance, assembly, and training. However, literature is scattered among different visual assets (i.e., AR visual user interface elements associated with a real scene). In this work, we present a systematic literature review of visual assets used in these industrial fields. We searched five databases, initially finding 1757 papers. Then, we selected 122 iAR papers from 1997 to 2019 and extracted 348 visual assets. We propose a classification for visual assets according to (i) what is displayed, (ii) how it conveys information (frame of reference, color coding, animation), and, (iii) why it is used. Our review shows that product models, text and auxiliary models are, in order, the most common, with each most often used to support operating, checking and locating tasks respectively. Other visual assets are scarcely used. Product and auxiliary models are commonly rendered world-fixed, color coding is not used as often as expected, while animations are limited to product and auxiliary model. This survey provides a snapshot of over 20 years of literature in iAR, useful to understand established practices to orientate in iAR interface design and to present future research directions.

Data stories often seek to elicit affective feelings from viewers. However, how to design affective data stories remains under-explored. In this work, we investigate one specific design factor, animation, and present Kineticharts, an animation design scheme for creating charts that express five positive affects: joy, amusement, surprise, tenderness, and excitement. These five affects were found to be frequently communicated through animation in data stories. Regarding each affect, we designed varied kinetic motions represented by bar charts, line charts, and pie charts, resulting in 60 animated charts for the five affects. We designed Kineticharts by first conducting a need-finding study with professional practitioners from data journalism and then analyzing a corpus of affective motion graphics to identify salient kinetic patterns. We evaluated Kineticharts through two user studies. The results suggest that Kineticharts can accurately convey affects, and improve the expressiveness of data stories, as well as enhance user engagement without hindering data comprehension compared to the animation design from DataClips, an authoring tool for data videos.

We present a CPU-based real-time cloth animation method for dressing virtual humans of various shapes and poses. Our approach formulates the clothing deformation as a high-dimensional function of body shape parameters and pose parameters. In order to accelerate the computation, our formulation factorizes the clothing deformation into two independent components: the deformation introduced by body pose variation (Clothing Pose Model) and the deformation from body shape variation (Clothing Shape Model). Furthermore, we sample and cluster the poses spanning the entire pose space and use those clusters to efficiently calculate the anchoring points. We also introduce a sensitivity-based distance measurement to both find nearby anchoring points and evaluate their contributions to the final animation. Given a query shape and pose of the virtual agent, we synthesize the resulting clothing deformation by blending the Taylor expansion results of nearby anchoring points. Compared to previous methods, our approach is general and able to add the shape dimension to any clothing pose model. Furthermore, we can animate clothing represented with tens of thousands of vertices at 50+ FPS on a CPU. We also conduct a user evaluation and show that our method can improve a user's perception of dressed virtual agents in an immersive virtual environment (IVE) compared to a realtime linear blend skinning method.

In this article, we present a data-driven approach for modeling and animation of 3D necks. Our method is based on a new neck animation model that decomposes the neck animation into local deformation caused by larynx motion and global deformation driven by head poses, facial expressions, and speech. A skinning model is introduced for modeling local deformation and underlying larynx motions, while the global neck deformation caused by each factor is modeled by its corrective blendshape set, respectively. Based on this neck model, we introduce a regression method to drive the larynx motion and neck deformation from speech. Both the neck model and the speech regressor are learned from a dataset of 3D neck animation sequences captured from different identities. Our neck model significantly improves the realism of facial animation and allows users to easily create plausible neck animations from speech and facial expressions. We verify our neck model and demonstrate its advantages in 3D neck tracking and animation.

Dynamic networks can be challenging to analyze visually, especially if they span a large time range during which new nodes and edges can appear and disappear. Although it is straightforward to provide interfaces for visualization that represent multiple states of the network (i.e., multiple timeslices) either simultaneously (e.g., through small multiples) or interactively (e.g., through interactive animation), these interfaces might not support tasks in which disjoint timeslices need to be compared. Since these tasks are key for understanding the dynamic aspects of the network, understanding which interactive visualizations best support these tasks is important. We present the results of a series of laboratory experiments comparing two traditional approaches (small multiples and interactive animation), with a more recent approach based on interactive timeslicing. The tasks were performed on a large display through a touch interface. Participants completed 24 trials of three tasks with all techniques. The results show that interactive timeslicing brings benefit when comparing distant points in time, but less benefits when analyzing contiguous intervals of time.

Data-driven modeling of human motions is ubiquitous in computer graphics and computer vision applications, such as synthesizing realistic motions or recognizing actions. Recent research has shown that such problems can be approached by learning a natural motion manifold using deep learning to address the shortcomings of traditional data-driven approaches. However, previous methods can be sub-optimal for two reasons. First, the skeletal information has not been fully utilized for feature extraction. Unlike images, it is difficult to define spatial proximity in skeletal motions in the way that deep networks can be applied. Second, motion is time-series data with strong multi-modal temporal correlations. A frame could be followed by several candidate frames leading to different motions; long-range dependencies exist where a number of frames in the beginning correlate to a number of frames later. Ineffective modeling would either under-estimate the multi-modality and variance, resulting in featureless mean motion or over-estimate them resulting in jittery motions. In this paper, we propose a new deep network to tackle these challenges by creating a natural motion manifold that is versatile for many applications. The network has a new spatial component for feature extraction. It is also equipped with a new batch prediction model that predicts a large number of frames at once, such that long-term temporally-based objective functions can be employed to correctly learn the motion multi-modality and variances. With our system, long-duration motions can be predicted/synthesized using an open-loop setup where the motion retains the dynamics accurately. It can also be used for denoising corrupted motions and synthesizing new motions with given control signals. We demonstrate that our system can create superior results comparing to existing work in multiple applications., 12 pages, Accepted in IEEE Transaction on Visualization and Computer Graphics

We introduce a new method for computing optimal path maps on the GPU using OpenGL shaders. Our method explores GPU rasterization as a way to propagate optimal costs on a polygonal 2D environment, producing optimal path maps which can efficiently be queried at run-time. Our method is implemented entirely with GPU shaders, does not require pre-computation, addresses optimal path maps with multiple points and line segments as sources, and introduces a new optimal path map concept not addressed before: maps with weights at vertices representing possible changes in traversal speed. The produced maps offer new capabilities not explored by previous navigation representations and at the same time address paths with global optimality, a characteristic which has been mostly neglected in animated virtual environments. The proposed path maps partition the input environment into the regions sharing a same parent point along the shortest path to the closest source, taking into account possible speed changes at vertices. The proposed approach is particularly suitable for the animation of multiple agents moving toward the entrances or exits of a virtual environment, a situation which is efficiently represented with the proposed path maps.

Traditional methods for motion comparison consider features from individual characters. However, the semantic meaning of many human activities is usually defined by the interaction between them, such as a high-five interaction of two characters. There is little success in adapting interaction-based features in activity comparison, as they either do not have a fixed topology or are in high dimensional. In this paper, we propose a unified framework for activity comparison from the interaction point of view. Our new metric evaluates the similarity of interaction by adapting the Earth Mover’s Distance onto a customized geometric mesh structure that represents spatial-temporal interactions. This allows us to compare different classes of interactions and discover their intrinsic semantic similarity. We created five interaction databases of different natures, covering both two characters (synthetic and real-people) and character-object interactions, which are open for public uses. We demonstrate how the proposed metric aligns well with the semantic meaning of the interaction. We also apply the metric in interaction retrieval and show how it outperforms existing ones. The proposed method can be used for unsupervised activity detection in monitoring systems and activity retrieval in smart animation systems.

Through avatar embodiment in Virtual Reality (VR) we can achieve the illusion that an avatar is substituting our body: the avatar moves as we move and we see it from a first person perspective. However, self-identification, the process of identifying a representation as being oneself, poses new challenges because a key determinant is that we see and have agency in our own face. Providing control over the face is hard with current HMD technologies because face tracking is either cumbersome or error prone. However, limited animation is easily achieved based on speaking. We investigate the level of avatar enfacement, that is believing that a picture of a face is one's own face, with three levels of facial animation: (i) one in which the facial expressions of the avatars are static, (ii) one in which we implement lip-sync motion and (iii) one in which the avatar presents lip-sync plus additional facial animations, with blinks, designed by a professional animator. We measure self-identification using a face morphing tool that morphs from the face of the participant to the face of a gender matched avatar. We find that self-identification on avatars can be increased through pre-baked animations even when these are not photorealistic nor look like the participant.

Given the expanding use of 3D Objects in a variety of fields such as animation, gaming, virtual worlds, commerce, augmented reality and 3D printing, we present a novel system for object browsing and searching. Specifically, the system packs objects into an interactive 3D cloud for browsing and searching. It was designed with the aim of increasing search efficiency in a variety of active environments, while providing a visually engaging layout, and we evaluated this by conducting a comparative user study. We show that our system can significantly decrease search times compared to the classic grid-based layout, and it has been suggested by subjects that cloud-based searching is more interesting and visually-engaging.

We introduce a framework for simulating a variety of nontrivial, socially motivated behaviors that underlie the orderly passage of pedestrians through doorways, especially the common courtesy of opening and holding doors open for others, an important etiquette that has been overlooked in the literature on autonomous multi-human animation. Emulating such social activity requires serious attention to the interplay of visual perception, navigation in constrained doorway environments, manipulation of a variety of door types, and high-level decision making based on social considerations. To tackle this complex human simulation problem, we take an artificial life approach to modeling autonomous pedestrians, proposing a layered architecture comprising mental, behavioral, and motor layers. The behavioral layer couples two stages: (1) a decentralized, agent-based strategy for dynamically determining the well-mannered ordering of pedestrians around doorways, and (2) a state-based model that directs and coordinates a pedestrian's interactions with the door. The mental layer is a Bayesian network decision model that dynamically selects appropriate door-holding behaviors by considering both internal and external social factors pertinent to pedestrians interacting with one another in and around doorways. Our framework addresses the various door types in common use and supports a variety of doorway etiquette scenarios with efficient, real-time performance.

Virtual traffic benefits a variety of applications, including video games, traffic engineering, autonomous driving, and virtual reality. To date, traffic visualization via different simulation models can reconstruct detailed traffic flows. However, each specific behavior of vehicles is always described by establishing an independent control model. Moreover, mutual interactions between vehicles and other road users are rarely modeled in existing simulators. An all-in-one simulator that considers the complex behaviors of all potential road users in a realistic urban environment is urgently needed. In this work, we propose a novel, extensible, and microscopic method to build heterogeneous traffic simulation using the force-based concept. This force-based approach can accurately replicate the sophisticated behaviors of various road users and their interactions in a simple and unified manner. We calibrate the model parameters using real-world traffic trajectory data. The effectiveness of this approach is demonstrated through many simulation experiments, as well as comparisons to real-world traffic data and popular microscopic simulators for traffic animation.

Autonomous driving technologies often use state-of-the-art artificial intelligence algorithms to understand the relationship between the vehicle and the external environment, to predict the changes of the environment, and then to plan and control the behaviors of the vehicle accordingly. The complexity of such technologies makes it challenging to evaluate the performance of autonomous driving systems and to find ways to improve them. The current approaches to evaluating such autonomous driving systems largely use a single score to indicate the overall performance of a system, but domain experts have difficulties in understanding how individual components or algorithms in an autonomous driving system may contribute to the score. To address this problem, we collaborate with domain experts on autonomous driving algorithms, and propose a visual evaluation method for autonomous driving. Our method considers the data generated in all components during the whole process of autonomous driving, including perception results, planning routes, prediction of obstacles, various controlling parameters, and evaluation of comfort. We develop a visual analytics workflow to integrate an evaluation mathematical model with adjustable parameters, support the evaluation of the system from the level of the overall performance to the level of detailed measures of individual components, and to show both evaluation scores and their contributing factors. Our implemented visual analytics system provides an overview evaluation score at the beginning and shows the animation of the dynamic change of the scores at each period. Experts can interactively explore the specific component at different time periods and identify related factors. With our method, domain experts not only learn about the performance of an autonomous driving system, but also identify and access the problematic parts of each component. Our visual evaluation system can be applied to the autonomous driving simulation system and used for various evaluation cases. The results of using our system in some simulation cases and the feedback from involved domain experts confirm the usefulness and efficiency of our method in helping people gain in-depth insight into autonomous driving systems.

We propose a novel framework for hair animation as well as hair-water interaction that supports millions of hairs. First, we develop a hair animation framework that embeds hair into a tetrahedralized volume mesh that we kinematically skin to deform and follow the exterior of an animated character. Allowing the hairs to follow their precomputed embedded locations in the kinematically deforming skinned mesh already provides visually plausible behavior. Creating a copy of the tetrahedral mesh, endowing it with springs, and attaching it to the kinematically skinned mesh creates more dynamic behavior. Notably, the springs can be quite weak and thus efficient to simulate because they are structurally supported by the kinematic mesh. If independent simulation of individual hairs or guide hairs is desired, they too benefit from being anchored to the kinematic mesh dramatically increasing efficiency as weak springs can be used while still supporting interesting and dramatic hairstyles. Furthermore, we explain how to embed these dynamic simulations into the kinematically deforming skinned mesh so that they can be used as part of a blendshape system where an artist can make many subsequent iterations without requiring any additional simulation. Although there are many applications for our newly proposed approach to hair animation, we mostly focus on the particularly challenging problem of hair-water interaction. While doing this, we discuss how porosities are stored in the kinematic mesh, how the kinematically deforming mesh can be used to apply drag and adhesion forces to the water, etc.

This paper introduces a novel technique for smooth and efficient zooming and panning based on dynamical systems in hyperbolic space. Unlike the technique of van Wijk and Nuij, the animations produced by our technique are smooth at the endpoints and when interrupted by a change of target. To analyze the results of our technique, we introduce world/screen diagrams, a novel technique for visualizing zooming and panning animations.

We examine the process of designing an exhibit to communicate scientific findings from a complex dataset and unfamiliar domain to the public in a science museum. Our exhibit sought to communicate new lessons based on scientific findings from the domain of metagenomics. This multi-user exhibit had three goals: (1) to inform the public about microbial communities and their daily cycles; (2) to link microbes' activity to the concept of gene expression; (3) and to highlight scientists' use of gene expression data to understand the role of microbes. To address these three goals, we derived visualization designs with three corresponding stories, each corresponding to a goal. We present three successive rounds of design and evaluation of our attempts to convey these goals. We could successfully present one story but had limited success with our second and third goals. This work presents a detailed account of an attempt to explain tightly coupled relationships through storytelling and animation in a multi-user, informal learning environment to a public with varying prior knowledge on the domain and identify lessons for future design.

Animated transitions help viewers follow changes between related visualizations. Specifying effective animations demands significant effort: authors must select the elements and properties to animate, provide transition parameters, and coordinate the timing of stages. To facilitate this process, we present Gemini, a declarative grammar and recommendation system for animated transitions between single-view statistical graphics. Gemini specifications define transition “steps” in terms of high-level visual components (marks, axes, legends) and composition rules to synchronize and concatenate steps. With this grammar, Gemini can recommend animation designs to augment and accelerate designers' work. Gemini enumerates staged animation designs for given start and end states, and ranks those designs using a cost function informed by prior perceptual studies. To evaluate Gemini, we conduct both a formative study on Mechanical Turk to assess and tune our ranking function, and a summative study in which 8 experienced visualization developers implement animations in D3 that we then compare to Gemini's suggestions. We find that most designs (9/11) are exactly replicable in Gemini, with many (8/11) achievable via edits to suggestions, and that Gemini suggestions avoid multiple participant errors.

Animated transitions can be effective in explaining and exploring a small number of visualizations where there are drastic changes in the scene over a short interval of time. This is especially true if data elements cannot be visually distinguished by other means. Current research in animated transitions has mainly focused on linear transitions (all elements follow straight line paths) or enhancing coordinated motion through bundling of linear trajectories. In this paper, we introduce animated transition design, a technique to build smooth, non-linear transitions for clustered data with either minimal or no user involvement. The technique is flexible and simple to implement, and has the additional advantage that it explicitly enhances coordinated motion and can avoid crowding, which are both important factors to support object tracking in a scene. We investigate its usability, provide preliminary evidence for the effectiveness of this technique through metric evaluations and user study and discuss limitations and future directions.

We present a real-time video stylization system and demonstrate a variety of painterly styles rendered on real video inputs. The key technical contribution lies on the object flow , which is robust to inaccurate optical flow, unknown object transformation and partial occlusion as well. Since object flows relate regions of the same object across frames, shower-door effect can be effectively reduced where painterly strokes and textures are rendered on video objects. The construction of object flows is performed in real time and automatically after applying metric learning. To reduce temporal flickering, we extend the bilateral filtering into motion bilateral filtering. We propose quantitative metrics to measure the temporal coherence on structures and textures of our stylized videos, and perform extensive experiments to compare our stylized results with baseline systems and prior works specializing in watercolor and abstraction.

We present a novel method for posing and animating botanical tree models interactively in real time. Unlike other state of the art methods which tend to produce trees that are overly flexible, bending and deforming as if they were underwater plants, our approach allows for arbitrarily high stiffness while still maintaining real-time frame rates without spurious artifacts, even on quite large trees with over ten thousand branches. This is accomplished by using an articulated rigid body model with as-stiff-as-desired rotational springs in conjunction with our newly proposed simulation technique, which is motivated both by position based dynamics and the typical $O(N)$ algorithms for articulated rigid bodies. The efficiency of our algorithm allows us to pose and animate trees with millions of branches or alternatively simulate a small forest comprised of many highly detailed trees. Even using only a single CPU core, we can simulate ten thousand branches in real time while still maintaining quite crisp user interactivity. This has allowed us to incorporate our framework into a commodity game engine to run interactively even on a low-budget tablet. We show that our method is amenable to the incorporation of a large variety of desirable effects such as wind, leaves, fictitious forces, collisions, fracture, etc.

We present DrawFromDrawings , an interactive drawing system that provides users with visual feedback for assistance in 2D drawing using a database of sketch images. Following the traditional imitation and emulation training from art education, DrawFromDrawings enables users to retrieve and refer to a sketch image stored in a database and provides them with various novel strokes as suggestive or deformation feedback. Given regions of interest (ROIs) in the user and reference sketches, DrawFromDrawings detects as-long-as-possible (ALAP) stroke segments and the correspondences between user and reference sketches that are the key to computing seamless interpolations. The stroke-level interpolations are parametrized with the user strokes, the reference strokes, and new strokes created by warping the reference strokes based on the user and reference ROI shapes, and the user study indicated that the interpolation could produce various reasonable strokes varying in shapes and complexity. DrawFromDrawings allows users to either replace their strokes with interpolated strokes (deformation feedback) or overlays interpolated strokes onto their strokes (suggestive feedback). The other user studies on the feedback modes indicated that the suggestive feedback enabled drawers to develop and render their ideas using their own stroke style, whereas the deformation feedback enabled them to finish the sketch composition quickly.

Reduced hair models have proven successful for interactively simulating a full head of hair strands, building upon a fundamental assumption that only a small set of guide hairs are needed for explicit simulation, and the rest of the hair move coherently and thus can be interpolated using guide hairs. Unfortunately, hair-solid interactions is a pathological case for traditional reduced hair models, as the motion coherence between hair strands can be arbitrarily broken by interacting with solids. In this paper, we propose an adaptive hair skinning method for interactive hair simulation with hair-solid collisions. We precompute many eligible sets of guide hairs and the corresponding interpolation relationships that are represented using a compact strand-based hair skinning model. At runtime, we simulate only guide hairs; for interpolating every other hair, we adaptively choose its guide hairs, taking into account motion coherence and potential hair-solid collisions. Further, we introduce a two-way collision correction algorithm to allow sparsely sampled guide hairs to resolve collisions with solids that can have small geometric features. Our method enables interactive simulation of more than 150 K hair strands interacting with complex solid objects, using 400 guide hairs. We demonstrate the efficiency and robustness of the method with various hairstyles and user-controlled arbitrary hair-solid interactions.

Data videos, or short data-driven motion graphics, are an increasingly popular medium for storytelling. However, creating data videos is difficult as it involves pulling together a unique combination of skills. We introduce DataClips, an authoring tool aimed at lowering the barriers to crafting data videos. DataClips allows non-experts to assemble data-driven "clips" together to form longer sequences. We constructed the library of data clips by analyzing the composition of over 70 data videos produced by reputable sources such as The New York Times and The Guardian. We demonstrate that DataClips can reproduce over 90% of our data videos corpus. We also report on a qualitative study comparing the authoring process and outcome achieved by (1) non-experts using DataClips, and (2) experts using Adobe Illustrator and After Effects to create data-driven clips. Results indicated that non-experts are able to learn and use DataClips with a short training period. In the span of one hour, they were able to produce more videos than experts using a professional editing tool, and their clips were rated similarly by an independent audience.

Glyph as a powerful multivariate visualization technique is used to visualize data through its visual channels. To visualize 3D volumetric dataset, glyphs are usually placed on 2D surface, such as the slicing plane or the feature surface, to avoid occluding each other. However, the 3D spatial structure of some features may be missing. On the other hand, placing large number of glyphs over the entire 3D space results in occlusion and visual clutter that make the visualization ineffective. To avoid the occlusion, we propose a view-dependent interactive 3D lens that removes the occluding glyphs by pulling the glyphs aside through the animation. We provide two space deformation models and two lens shape models to displace the glyphs based on their spatial distributions. After the displacement, the glyphs around the user-interested region are still visible as the context information, and their spatial structures are preserved. Besides, we attenuate the brightness of the glyphs inside the lens based on their depths to provide more depth cue. Furthermore, we developed an interactive glyph visualization system to explore different glyph-based visualization applications. In the system, we provide a few lens utilities that allows users to pick a glyph or a feature and look at it from different view directions. We compare different display/interaction techniques to visualize/manipulate our lens and glyphs.

Spatial Augmented Reality (SAR) systems can be suitably combined with other existing Extended Reality (xR) technologies to support collaboration. In existing strategies, users unencumbered by a viewing technology, such as a tablet interface or a head-mounted display, must rely on the transmission of their collaborators' positioning through interpreting a first-person camera view. This design creates a seam between a user's experience of the augmented physical environment in SAR, and their collaborators' experience inside the virtual environment. To assist in development and evaluation of spatial cues to support spatial awareness in SAR environments, an egocentric spatial-communication taxonomy is presented given two determining dimensions, a cue's attachment (physical/virtual) and animation (local/world). We developed four egocentric cues which characterize the four independent dimensions of the matrix: arrow, path, glow, and radial, and a single exocentric world in miniature visualization. Our study shows that virtual attachment cues are preferred, providing the highest accuracy, highest performance when collaborators are occluded, and produce the least mental effort when used with a single virtual collaborator. For multiple collaborators however, the virtual attached, world animated radial cue produces significant increases in mental load and reductions in preference, demonstrating the impact of visual augmentation clutter. The single exocentric visualization produced higher levels of head movement, and poorer accuracy, however the novelty of the visualization produced positive qualitative results. Refereed/Peer-reviewed

Rendering in virtual reality (VR) requires substantial computational power to generate 90 frames per second at high resolution with good-quality antialiasing. The video data sent to a VR headset requires high bandwidth, achievable only on dedicated links. In this paper we explain how rendering requirements and transmission bandwidth can be reduced using a conceptually simple technique that integrates well with existing rendering pipelines. Every even-numbered frame is rendered at a lower resolution, and every odd-numbered frame is kept at high resolution but is modified in order to compensate for the previous loss of high spatial frequencies. When the frames are seen at a high frame rate, they are fused and perceived as high-resolution and high-frame-rate animation. The technique relies on the limited ability of the visual system to perceive high spatio-temporal frequencies. Despite its conceptual simplicity, correct execution of the technique requires a number of non-trivial steps: display photometric temporal response must be modeled, flicker and motion artifacts must be avoided, and the generated signal must not exceed the dynamic range of the display. Our experiments, performed on a high-frame-rate LCD monitor and OLED-based VR headsets, explore the parameter space of the proposed technique and demonstrate that its perceived quality is indistinguishable from full-resolution rendering. The technique is an attractive alternative to reprojection and resolution reduction of all frames.

International audience; This paper introduces dynamic composite physicalizations, a new class of physical visualizations that use collections of self-propelled objects to represent data. Dynamic composite physicalizations can be used both to give physical form to well-known interactive visualization techniques, and to explore new visualizations and interaction paradigms. We first propose a design space characterizing composite physicalizations based on previous work in the fields of Information Visualization and Human Computer Interaction. We illustrate dynamic composite physicalizations in two scenarios demonstrating potential benefits for collaboration and decision making, as well as new opportunities for physical interaction. We then describe our implementation using wheeled micro-robots capable of locating themselves and sensing user input, before discussing limitations and opportunities for future work.

This paper presents a declarative grammar for conveniently and effectively specifying advanced volume visualizations. Existing methods for creating volume visualizations either lack the flexibility to specify sophisticated visualizations or are difficult to use for those unfamiliar with volume rendering implementation and parameterization. Our design provides the ability to quickly create expressive visualizations without knowledge of the volume rendering implementation. It attempts to capture aspects of those difficult but powerful methods while remaining flexible and easy to use. As a proof of concept, our current implementation of the grammar allows users to combine multiple data variables in various ways and define transfer functions for diverse input data. The grammar also has the ability to describe advanced shading effects and create animations. We demonstrate the power and flexibility of our approach using multiple practical volume visualizations.

International audience; Before-and-after image pairs show how entities in a given region have evolved over a specific period of time. Satellite images are a major source of such data, that capture how natural phenomena or human activity impact a geographical area. These images are used both for data analysis and to illustrate the resulting findings to diverse audiences. The simple techniques used to display them, including juxtaposing, swapping and monolithic blending, often fail to convey the underlying phenomenon in a meaningful manner. We introduce Baia, a framework to create advanced animated transitions, called animation plans, between before-and-after images. Baia relies on a pixel-based transition model that gives authors much expressive power, while keeping animations for common types of changes easy to create thanks to predefined animation primitives. We describe our model, the associated animation editor, and report on two user studies. In the first study, advanced transitions enabled by Baia were compared to monolithic blending, and perceived as more realistic and better at focusing viewer's attention on a region of interest than the latter. The second study aimed at gathering feedback about the usability of Baia's animation editor.

We propose Average Vector Field (AVF) integration for simulation of deformable solids in physics-based animation. Our method achieves exact energy conservation for the St. Venant-Kirchhoff material without any correction steps or extra parameters. Exact energy conservation implies that our resulting animations 1) cannot explode and 2) do not suffer from numerical damping, which are two common problems with previous numerical integration techniques. Our method produces lively motion even with large time steps as typically used in physics-based animation. Our implicit update rules can be formulated as a minimization problem and solved in a similar way as optimization-based backward Euler, with only a mild computing overhead. Our approach also supports damping and collision response models, making it easy to deploy in practical computer animation pipelines.

This paper addresses a challenging single-view modeling and animation problem with cartoon images. Our goal is to model the hairs in a given cartoon image with consistent layering and occlusion, so that we can produce various visual effects from just a single image. We propose a novel 2.5D modeling approach to deal with this problem. Given an input image, we first segment the hairs of the cartoon character into regions of hair strands. Then, we apply our novel layering metric, which is derived from the Gestalt psychology, to automatically optimize the depth ordering among the hair strands. After that, we employ our hair completion method to fill the occluded part of each hair strand, and create a 2.5D model of the cartoon hair. By using this model, we can produce various visual effects, e.g., we develop a simplified fluid simulation model to produce wind blowing animations with the 2.5D hairs. To further demonstrate the applicability and versatility of our method, we compare our results with real cartoon hair animations, and also apply our model to produce a wide variety of hair manipulation effects, including hair editing and hair braiding.

Interactive visual applications often rely on animation to transition from one display state to another. There are multiple animation techniques to choose from, and it is not always clear which should produce the best visual correspondences between display elements. One major factor is whether the animation relies on staggering-an incremental delay in start times across the moving elements. It has been suggested that staggering may reduce occlusion, while also reducing display complexity and producing less overwhelming animations, though no empirical evidence has demonstrated these advantages. Work in perceptual psychology does show that reducing occlusion, and reducing inter-object proximity (crowding) more generally, improves performance in multiple object tracking. We ran simulations confirming that staggering can in some cases reduce crowding in animated transitions involving dot clouds (as found in, e.g., animated 2D scatterplots). We empirically evaluated the effect of two staggering techniques on tracking tasks, focusing on cases that should most favour staggering. We found that introducing staggering has a negligible, or even negative, impact on multiple object tracking performance. The potential benefits of staggering may be outweighed by strong costs: a loss of common-motion grouping information about which objects travel in similar paths, and less predictability about when any specific object would begin to move. Staggering may be beneficial in some conditions, but they have yet to be demonstrated. The present results are a significant step toward a better understanding of animation pacing, and provide direction for further research.

We present a novel framework for visualizing routes on mobile devices. Our framework is suitable for helping users explore their environment. First, given a starting point and a maximum route length, the system retrieves nearby points of interest (POIs). Second, we automatically compute an attractive walking path through the environment trying to pass by as many highly ranked POIs as possible. Third, we automatically compute a route visualization that shows the current user position, POI locations via pins, and detail lenses for more information about the POIs. The visualization is an animation of an orthographic map view that follows the current user position. We propose an optimization based on a binary integer program (BIP) that models multiple requirements for an effective placement of detail lenses. We show that our path computation method outperforms recently proposed methods and we evaluate the overall impact of our framework in two user studies.

We present a system to combine arbitrary triangle mesh animations with physically based Finite Element Method (FEM) simulation, enabling control over the combination both in space and time. The input is a triangle mesh animation obtained using any method, such as keyframed animation, character rigging, 3D scanning, or geometric shape modeling. The input may be non-physical, crude or even incomplete. The user provides weights, specified using a minimal user interface, for how much physically based simulation should be allowed to modify the animation in any region of the model, and in time. Our system then computes a physically-based animation that is constrained to the input animation to the amount prescribed by these weights. This permits smoothly turning physics on and off over space and time, making it possible for the output to strictly follow the input, to evolve purely based on physically based simulation, and anything in between. Achieving such results requires a careful combination of several system components. We propose and analyze these components, including proper automatic creation of simulation meshes (even for non-manifold and self-colliding undeformed triangle meshes), converting triangle mesh animations into animations of the simulation mesh, and resolving collisions and self-collisions while following the input.

Remote visualization has become both a necessity, as data set sizes have grown faster than computer network performance, and an opportunity, as laptop, tablet, and smartphone mobile computing platforms have become ubiquitous. However, the conventional remote visualization (CRV) approach of sending a new image from the server to the client for every view parameter change suffers from reduced interactivity. One problem is high latency, as the network has to be traversed twice, once to communicate the view parameters to the server and once to transmit the new image to the client. A second problem is reduced image quality due to aggressive compression or low resolution. We address these problems by constructing and transmitting enhanced images that are sufficient for quality output frame reconstruction at the client for a range of view parameter values. The client reconstructs thousands of frames locally, without any additional data from the server, which avoids latency and aggressive compression. We introduce animated depth images, which not only store a color and depth sample at every pixel, but also store the trajectory of the samples for a given time interval. Sample trajectories are stored compactly by partitioning the image into semi-rigid sample clusters and by storing one sequence of rigid body transformations per cluster. Animated depth images leverage sample trajectory coherence to achieve a good compression of animation data, with a small and user-controllable approximation error. We demonstrate animated depth images in the context of finite element analysis and SPH data sets.

We present ADAPT, a flexible platform for designing and authoring functional, purposeful human characters in a rich virtual environment. Our framework incorporates character animation, navigation, and behavior with modular interchangeable components to produce narrative scenes. Our animation system provides locomotion, reaching, gaze tracking, gesturing, sitting, and reactions to external physical forces, and can easily be extended with more functionality due to a decoupled, modular structure. Additionally, our navigation component allows characters to maneuver through a complex environment with predictive steering for dynamic obstacle avoidance. Finally, our behavior framework allows a user to fully leverage a character's animation and navigation capabilities when authoring both individual decision-making and complex interactions between actors using a centralized, event-driven model.

International audience; Identifying, tracking and understanding changes in dynamic networks are complex and cognitively demanding tasks. We present GraphDiaries, a visual interface designed to improve support for these tasks in any node-link based graph visualization system. GraphDiaries relies on animated transitions that highlight changes in the network between time steps, thus helping users identify and understand those changes. To better understand the tasks related to the exploration of dynamic networks, we first introduce a task taxonomy, that informs the design of GraphDiaries, presented afterwards. We then report on a user study, based on representative tasks identified through the taxonomy, and that compares GraphDiaries to existing techniques for temporal navigation in dynamic networks, showing that it outperforms them in terms of both task time and errors for several of these tasks.

International audience; In this paper we study how the visual animation of a self-avatar can be artificially modified in real-time in order to generate different haptic perceptions. In our experimental setup, participants could watch their self-avatar in a virtual environment in mirror mode while performing a weight lifting task. Users could map their gestures on the self-animated avatar in real-time using a Kinect. We introduce three kinds of modification of the visual animation of the self-avatar according to the effort delivered by the virtual avatar: 1) changes on the spatial mapping between the user's gestures and the avatar, 2) different motion profiles of the animation, and 3) changes in the posture of the avatar (upper-body inclination). The experimental task consisted of a weight lifting task in which participants had to order four virtual dumbbells according to their virtual weight. The user had to lift each virtual dumbbells by means of a tangible stick, the animation of the avatar was modulated according to the virtual weight of the dumbbell. The results showed that the altering the spatial mapping delivered the best performance. Nevertheless, participants globally appreciated all the different visual effects. Our results pave the way to the exploitation of such novel techniques in various VR applications such as sport training, exercise games, or industrial training scenarios in single or collaborative mode.

We empirically examined the impact of virtual human animation on the emotional responses of participants in a medical virtual reality system for education in the signs and symptoms of patient deterioration. Participants were presented with one of two virtual human conditions in a between-subjects experiment, static (non-animated) and dynamic (animated). Our objective measures included the use of psycho-physical Electro Dermal Activity (EDA) sensors, and subjective measures inspired by social psychology research included the Differential Emotions Survey (DES IV) and Positive and Negative Affect Survey (PANAS). We analyzed the quantitative and qualitative measures associated with participants#146; emotional state at four distinct time-steps in the simulated interpersonal experience as the virtual patient#146;s medical condition deteriorated. Results suggest that participants in the dynamic condition with animations exhibited a higher sense of co-presence and greater emotional response as compared to participants in the static condition, corresponding to the deterioration in the medical condition of the virtual patient. Negative affect of participants in the dynamic condition increased at a higher rate than for participants in the static condition. The virtual human animations elicited a stronger response in negative emotions such as anguish, fear, and anger as the virtual patient#146;s medical condition worsened.

Virtual reality (VR) has been successfully applied to a broad range of training domains; however, to date there is little research investigating its benefits for sport psychology training. We hypothesized that using high-fidelity VR systems to display realistic 3D sport environments could trigger anxiety, allowing resilience-training systems to prepare athletes for real-world, high-pressure situations. In this work we investigated the feasibility and usefulness of using VR for sport psychology training. We developed a virtual soccer goalkeeping application for the Virginia Tech Visionarium VisCube (a CAVE-like display system), in which users defend against simulated penalty kicks using their own bodies. Using the application, we ran a controlled, within-subjects experiment with three independent variables: known anxiety triggers, field of regard, and simulation fidelity. The results demonstrate that a VR sport-oriented system can induce increased anxiety (physiological and subjective measures) compared to a baseline condition. There were a number of main effects and interaction effects for all three independent variables in terms of the subjective measures of anxiety. Both known anxiety triggers and simulation fidelity had a direct relationship to anxiety, while field of regard had an inverse relationship. Overall, the results demonstrate great potential for VR sport psychology training systems; however, further research is needed to determine if training in a VR environment can lead to long-term reduction in sport-induced anxiety.

The degrees of freedom of a crowd is much higher than that provided by a standard user input device. Typically, crowd-control systems require multiple passes to design crowd movements by specifying waypoints, and then defining character trajectories and crowd formation. Such multi-pass control would spoil the responsiveness and excitement of real-time control systems. In this paper, we propose a single-pass algorithm to control a crowd in complex environments. We observe that low-level details in crowd movement are related to interactions between characters and the environment, such as diverging/merging at cross points, or climbing over obstacles. Therefore, we simplify the problem by representing the crowd with a deformable mesh, and allow the user, via multitouch input, to specify high-level movements and formations that are important for context delivery. To help prevent congestion, our system dynamically reassigns characters in the formation by employing a mass transport solver to minimize their overall movement. The solver uses a cost function to evaluate the impact from the environment, including obstacles and areas affecting movement speed. Experimental results show realistic crowd movement created with minimal high-level user inputs. Our algorithm is particularly useful for real-time applications including strategy games and interactive animation creation.

Biologists often use computer graphics to visualize structures, which due to physical limitations are not possible to image with a microscope. One example for such structures are microtubules, which are present in every eukaryotic cell. They are part of the cytoskeleton maintaining the shape of the cell and playing a key role in the cell division. In this paper, we propose a scientifically-accurate multi-scale procedural model of microtubule dynamics as a novel application scenario for procedural animation, which can generate visualizations of their overall shape, molecular structure, as well as animations of the dynamic behaviour of their growth and disassembly. The model is spanning from tens of micrometers down to atomic resolution. All the aspects of the model are driven by scientific data. The advantage over a traditional, manual animation approach is that when the underlying data change, for instance due to new evidence, the model can be recreated immediately. The procedural animation concept is presented in its generic form, with several novel extensions, facilitating an easy translation to other domains with emergent multi-scale behavior.

Presenting and communicating insights to an audience-telling a story-is one of the main goals of data exploration. Even though visualization as a storytelling medium has recently begun to gain attention, storytelling is still underexplored in information visualization and little research has been done to help people tell their stories with data. To create a new, more engaging form of storytelling with data, we leverage and extend the narrative storytelling attributes of whiteboard animation with pen and touch interactions. We present SketchStory, a data-enabled digital whiteboard that facilitates the creation of personalized and expressive data charts quickly and easily. SketchStory recognizes a small set of sketch gestures for chart invocation, and automatically completes charts by synthesizing the visuals from the presenter-provided example icon and binding them to the underlying data. Furthermore, SketchStory allows the presenter to move and resize the completed data charts with touch, and filter the underlying data to facilitate interactive exploration. We conducted a controlled experiment for both audiences and presenters to compare SketchStory with a traditional presentation system, Microsoft PowerPoint. Results show that the audience is more engaged by presentations done with SketchStory than PowerPoint. Eighteen out of 24 audience participants preferred SketchStory to PowerPoint. Four out of five presenter participants also favored SketchStory despite the extra effort required for presentation.

Distributed systems are complex to develop and administer, and performance problem diagnosis is particularly challenging. When performance degrades, the problem might be in any of the system's many components or could be a result of poor interactions among them. Recent research efforts have created tools that automatically localize the problem to a small number of potential culprits, but research is needed to understand what visualization techniques work best for helping distributed systems developers understand and explore their results. This paper compares the relative merits of three well-known visualization approaches (side-by-side, diff, and animation) in the context of presenting the results of one proven automated localization technique called request-flow comparison. Via a 26-person user study, which included real distributed systems developers, we identify the unique benefits that each approach provides for different problem types and usage modes.

We present MotionExplorer, an exploratory search and analysis system for sequences of human motion in large motion capture data collections. This special type of multivariate time series data is relevant in many research fields including medicine, sports and animation. Key tasks in working with motion data include analysis of motion states and transitions, and synthesis of motion vectors by interpolation and combination. In the practice of research and application of human motion data, challenges exist in providing visual summaries and drill-down functionality for handling large motion data collections. We find that this domain can benefit from appropriate visual retrieval and analysis support to handle these tasks in presence of large motion data. To address this need, we developed MotionExplorer together with domain experts as an exploratory search system based on interactive aggregation and visualization of motion states as a basis for data navigation, exploration, and search. Based on an overview-first type visualization, users are able to search for interesting sub-sequences of motion based on a query-by-example metaphor, and explore search results by details on demand. We developed MotionExplorer in close collaboration with the targeted users who are researchers working on human motion synthesis and analysis, including a summative field study. Additionally, we conducted a laboratory design study to substantially improve MotionExplorer towards an intuitive, usable and robust design. MotionExplorer enables the search in human motion capture data with only a few mouse clicks. The researchers unanimously confirm that the system can efficiently support their work.

This paper presents a simple, three stage method to simulate the mechanics of wetting of porous solid objects, like sponges and cloth, when they interact with a fluid. In the first stage, we model the absorption of fluid by the object when it comes in contact with the fluid. In the second stage, we model the transport of absorbed fluid inside the object, due to diffusion, as a flow in a deforming, unstructured mesh. The fluid diffuses within the object depending on saturation of its various parts and other body forces. Finally, in the third stage, oversaturated parts of the object shed extra fluid by dripping. The simulation model is motivated by the physics of imbibition of fluids into porous solids in the presence of gravity. It is phenomenologically capable of simulating wicking and imbibition, dripping, surface flows over wet media, material weakening, and volume expansion due to wetting. The model is inherently mass conserving and works for both thin 2D objects like cloth and for 3D volumetric objects like sponges. It is also designed to be computationally efficient and can be easily added to existing cloth, soft body, and fluid simulation pipelines.

This paper presents the first method for full-body trajectory optimization of physics-based human motion that does not rely on motion capture, specified key-poses, or periodic motion. Optimization is performed using a small set of simple goals, for example, one hand should be on the ground, or the center-of-mass should be above a particular height. These objectives are applied to short spacetime windows which can be composed to express goals over an entire animation. Specific contact locations needed to achieve objectives are not required by our method. We show that the method can synthesize many different kinds of movement, including walking, hand walking, breakdancing, flips, and crawling. Most of these movements have never been previously synthesized by physics-based methods.