Search

The growing complexity of spatial and structural information in 3D data makes data inspection and visualization a challenging task. We describe a method to create a planar embedding of 3D treelike structures using their skeleton representations. Our method maintains the original geometry, without overlaps, to the best extent possible, allowing exploration of the topology within a single view. We present a novel camera view generation method which maximizes the visible geometric attributes (segment shape and relative placement between segments). Camera views are created for individual segments and are used to determine local bending angles at each node by projecting them to 2D. The final embedding is generated by minimizing an energy function (the weights of which are user adjustable) based on branch length and the 2D angles, while avoiding intersections. The user can also interactively modify segment placement within the 2D embedding, and the overall embedding will update accordingly. A global to local interactive exploration is provided using hierarchical camera views that are created for subtrees within the structure. We evaluate our method both qualitatively and quantitatively and demonstrate our results by constructing planar visualizations of line data (traced neurons) and volume data (CT vascular and bronchial data).

In this work, we present a novel method called WSDesc to learn 3D local descriptors in a weakly supervised manner for robust point cloud registration. Our work builds upon recent 3D CNN-based descriptor extractors, which leverage a voxel-based representation to parameterize local geometry of 3D points. Instead of using a predefined fixed-size local support in voxelization, we propose to learn the optimal support in a data-driven manner. To this end, we design a novel differentiable voxelization layer that can back-propagate the gradient to the support size optimization. To train the extracted descriptors, we propose a novel registration loss based on the deviation from rigidity of 3D transformations, and the loss is weakly supervised by the prior knowledge that the input point clouds have partial overlap, without requiring ground-truth alignment information. Through extensive experiments, we show that our learned descriptors yield superior performance on existing geometric registration benchmarks., Comment: To appear in IEEE TVCG

This paper first presents CycleDance, a novel dance style transfer system that transforms an existing motion clip in one dance style into a motion clip in another dance style while attempting to preserve the motion context of the dance. CycleDance extends existing CycleGAN architectures with multimodal transformer encoders to account for the music context. We adopt a sequence length-based curriculum learning strategy to stabilize training. Our approach captures rich and long-term intra-relations between motion frames, which is a common challenge in motion transfer and synthesis work. Building upon CycleDance, we further propose StarDance, which enables many-to-many mappings across different styles using a single generator network. Additionally, we introduce new metrics for gauging transfer strength and content preservation in the context of dance movements. To evaluate the performance of our approach, we perform an extensive ablation study and a human study with 30 participants, each with 5 or more years of dance experience. Our experimental results show that our approach can generate realistic movements with the target style, outperforming the baseline CycleGAN and its variants on naturalness, transfer strength, and content preservation. Our proposed approach has potential applications in choreography, gaming, animation, and tool development for artistic and scientific innovations in the field of dance.

Applications like physics, medicine, earth sciences, mechanical engineering, geo-engineering, bio-engineering use tensorial data. For example, tensors are used in formulating the balance equations of charge, mass, momentum, or energy as well as the constitutive relations that complement them. Some of these tensors (i.e. stiffness tensor, strain gradient, photo-elastic tensor) are of order higher than two. Currently, there are nearly no visualization techniques for such data beyond glyphs. An important reason for this is the limit of currently used tensor decomposition techniques. In this article, we propose to use the deviatoric decomposition to draw lines describing tensors of arbitrary order in three dimensions. The deviatoric decomposition splits a three-dimensional tensor of any order with any type of index symmetry into totally symmetric, traceless tensors. These tensors, called deviators, can be described by a unique set of directions (called multipoles by J. C. Maxwell) and scalars. These multipoles allow the definition of multipole lines which can be computed in a similar fashion to tensor lines and allow a line-based visualization of three-dimensional tensors of any order. We give examples for the visualization of symmetric, second-order tensor fields as well as fourth-order tensor fields. To allow an interpretation of the multipole lines, we analyze the connection between the multipoles and the eigenvectors/eigenvalues in the second-order case. For the fourth-order stiffness tensor, we prove relations between multipoles and the eigenvectors of the second-order right Cauchy-Green tensor and present different interpretations of the multipole lines.

We present a differentiable formulation of rigid-body contact dynamics for objects and robots represented as compositions of convex primitives. Existing optimization-based approaches simulating contact between convex primitives rely on a bilevel formulation that separates collision detection and contact simulation. These approaches are unreliable in realistic contact simulation scenarios because isolating the collision detection problem introduces contact location non-uniqueness. Our approach combines contact simulation and collision detection into a unified single-level optimization problem. This disambiguates the collision detection problem in a physics-informed manner. Compared to previous differentiable simulation approaches, our formulation features improved simulation robustness and a reduction in computational complexity by more than an order of magnitude. We illustrate the contact and collision differentiability on a robotic manipulation task requiring optimization-through-contact. We provide a numerically efficient implementation of our formulation in the Julia language called Silico.jl.

Network graphs are common visualization charts. They often appear in the form of bitmaps in papers, web pages, magazine prints, and designer sketches. People often want to modify network graphs because of their poor design, but it is difficult to obtain their underlying data. In this paper, we present VividGraph, a pipeline for automatically extracting and redesigning network graphs from static images. We propose using convolutional neural networks to solve the problem of network graph data extraction. Our method is robust to hand-drawn graphs, blurred graph images, and large graph images. We also present a network graph classification module to make it effective for directed graphs. We propose two evaluation methods to demonstrate the effectiveness of our approach. It can be used to quickly transform designer sketches, extract underlying data from existing network graphs, and interactively redesign poorly designed network graphs.

We present the results of a scientometric analysis of 30 years of IEEE VIS publications between 1990-2020, in which we conducted a multifaceted analysis of interdisciplinary collaboration and gender composition among authors. To this end, we curated BiblioVIS, a bibliometric dataset that contains rich metadata about IEEE VIS publications, including 3032 papers and 6113 authors. One of the main factors differentiating BiblioVIS from similar datasets is the authors' gender and discipline data, which we inferred through iterative rounds of computational and manual processes. Our analysis shows that, by and large, inter-institutional and interdisciplinary collaboration has been steadily growing over the past 30 years. However, interdisciplinary research was mainly between a few fields, including Computer Science, Engineering and Technology, and Medicine and Health disciplines. Our analysis of gender shows steady growth in women's authorship. Despite this growth, the gender distribution is still highly skewed, with men dominating (~75%) of this space. Our predictive analysis of gender balance shows that if the current trends continue, gender parity in the visualization field will not be reached before the third quarter of the century (~2070). Our primary goal in this work is to call the visualization community's attention to the critical topics of collaboration, diversity, and gender. Our research offers critical insights through the lens of diversity and gender to help accelerate progress towards a more diverse and representative research community.

Feature related particle data analysis plays an important role in many scientific applications such as fluid simulations, cosmology simulations and molecular dynamics. Compared to conventional methods that use hand-crafted feature descriptors, some recent studies focus on transforming the data into a new latent space, where features are easier to be identified, compared and extracted. However, it is challenging to transform particle data into latent representations, since the convolution neural networks used in prior studies require the data presented in regular grids. In this paper, we adopt Geometric Convolution, a neural network building block designed for 3D point clouds, to create latent representations for scientific particle data. These latent representations capture both the particle positions and their physical attributes in the local neighborhood so that features can be extracted by clustering in the latent space, and tracked by applying tracking algorithms such as mean-shift. We validate the extracted features and tracking results from our approach using datasets from three applications and show that they are comparable to the methods that define hand-crafted features for each specific dataset.

Unmanned Aerial Vehicles (UAVs) exhibit great agility but usually require an experienced pilot to operate them in certain applications such as inspection for disaster scenarios or buildings. The reduction of cognitive overload when driving this kind of aerial robot becomes a challenge and several solutions can be found in the literature. A new virtual control scheme for reducing this cognitive overload when controlling an aerial robot is proposed in this paper. The architecture is based on a novel interaction Drone Exocentric Advanced Metaphor (DrEAM) located in a Cave Automated Virtual Environment (CAVE) and a real robot containing an embedded controller based on quaternion formulation. The testing room, where real robots are evolving, is located away from the CAVE and they are connected via UDP in a ground station. The user controls manually a virtual drone through the DrEAM interaction metaphor, and the real robot imitates autonomously in real time the trajectory imposed by the user in the virtual environment. Experimental results illustrate the easy implementation and feasibility of the proposed scheme in two different scenarios. Results from these tests show that the mental effort when controlling a drone using the proposed virtual control scheme is lower than when controlling it in direct view. Moreover, the easy maneuverability and controllability of the real drone is also demonstrated in real time experiments.

Class-agnostic counting is usually phrased as a matching problem between a user-defined exemplar patch and a query image. The count is derived based on the number of objects similar to the exemplar patch. However, defining a target class using only positive exemplar patches inevitably miscounts unintended objects that are visually alike to the exemplar. In this paper, we propose to include negative exemplars that define what not to count. This allows the model to calibrate its notion of what is similar based on both positive and negative exemplars. It effectively disentangles visually similar negatives, leading to a more discriminative definition of the target object. We designed our method such that it can be incorporated with other class-agnostic counting models. Moreover, application-wise, our model can be used into a semi-automatic labeling tool to simplify the job of the annotator

Head tracking in head-mounted displays (HMDs) enables users to explore a 360-degree virtual scene with free head movements. However, for seated use of HMDs such as users sitting on a chair or a couch, physically turning around 360-degree is not possible. Redirection techniques decouple tracked physical motion and virtual motion, allowing users to explore virtual environments with more flexibility. In seated situations with only head movements available, the difference of stimulus might cause the detection thresholds of rotation gains to differ from that of redirected walking. Therefore we present an experiment with a two-alternative forced-choice (2AFC) design to compare the thresholds for seated and standing situations. Results indicate that users are unable to discriminate rotation gains between 0.89 and 1.28, a smaller range compared to the standing condition. We further treated head amplification as an interaction technique and found that a gain of 2.5, though not a hard threshold, was near the largest gain that users consider applicable. Overall, our work aims to better understand human perception of rotation gains in seated VR and the results provide guidance for future design choices of its applications., Comment: Submitted to IEEE TVCG

In this paper, we introduce a new framework for unsupervised deep homography estimation. Our contributions are 3 folds. First, unlike previous methods that regress 4 offsets for a homography, we propose a homography flow representation, which can be estimated by a weighted sum of 8 pre-defined homography flow bases. Second, considering a homography contains 8 Degree-of-Freedoms (DOFs) that is much less than the rank of the network features, we propose a Low Rank Representation (LRR) block that reduces the feature rank, so that features corresponding to the dominant motions are retained while others are rejected. Last, we propose a Feature Identity Loss (FIL) to enforce the learned image feature warp-equivariant, meaning that the result should be identical if the order of warp operation and feature extraction is swapped. With this constraint, the unsupervised optimization can be more effective and the learned features are more stable. With global-to-local homography flow refinement, we also naturally generalize the proposed method to local mesh-grid homography estimation, which can go beyond the constraint of a single homography. Extensive experiments are conducted to demonstrate the effectiveness of all the newly proposed components, and results show that our approach outperforms the state-of-the-art on the homography benchmark dataset both qualitatively and quantitatively. Code is available at https://github.com/megvii-research/BasesHomo.

Businesses in high-risk environments have been reluctant to adopt modern machine learning approaches due to their complex and uninterpretable nature. Most current solutions provide local, instance-level explanations, but this is insufficient for understanding the model as a whole. In this work, we show that strategy clusters (i.e., groups of data instances that are treated distinctly by the model) can be used to understand the global behavior of a complex ML model. To support effective exploration and understanding of these clusters, we introduce StrategyAtlas, a system designed to analyze and explain model strategies. Furthermore, it supports multiple ways to utilize these strategies for simplifying and improving the reference model. In collaboration with a large insurance company, we present a use case in automatic insurance acceptance, and show how professional data scientists were enabled to understand a complex model and improve the production model based on these insights.