Search

This paper proposes ShapeShifter, a new 3D generative model that learns to synthesize shape variations based on a single reference model. While generative methods for 3D objects have recently attracted much attention, current techniques often lack geometric details and/or require long training times and large resources. Our approach remedies these issues by combining sparse voxel grids and point, normal, and color sampling within a multiscale neural architecture that can be trained efficiently and in parallel. We show that our resulting variations better capture the fine details of their original input and can handle more general types of surfaces than previous SDF-based methods. Moreover, we offer interactive generation of 3D shape variants, allowing more human control in the design loop if needed.

Bayesian inference is conceptually elegant, but calculating posterior expectations can entail a heavy computational cost. Monte Carlo methods are reliable and supported by strong asymptotic guarantees, but do not leverage smoothness of the integrand. Solving Stein equations has emerged as a possible alternative, providing a framework for numerical approximation of posterior expectations in which smoothness can be exploited. However, existing numerical methods for Stein equations are associated with high computational cost due to the need to solve large linear systems. This paper considers the combination of iterative linear solvers and preconditioning strategies to obtain fast approximate solutions of Stein equations. Our main finding is that these methods can be effective, but that the performance of different preconditioning strategies is context-dependent, so that no single strategy can be universally preferred.

Recent probabilistic methods for 3D triangular meshes capture diverse shapes by differentiable mesh connectivity, but face high computational costs with increased shape details. We introduce a new differentiable mesh processing method in 2D and 3D that addresses this challenge and efficiently handles meshes with intricate structures. Additionally, we present an algorithm that adapts the mesh resolution to local geometry in 2D for efficient representation. We demonstrate the effectiveness of our approach on 2D point cloud and 3D multi-view reconstruction tasks. Visit our project page (https://sonsang.github.io/dmesh2-project) for source code and supplementary material., Comment: 26 pages, 27 figures, 4 tables

Pattern images are everywhere in the digital and physical worlds, and tools to edit them are valuable. But editing pattern images is tricky: desired edits are often programmatic: structure-aware edits that alter the underlying program which generates the pattern. One could attempt to infer this underlying program, but current methods for doing so struggle with complex images and produce unorganized programs that make editing tedious. In this work, we introduce a novel approach to perform programmatic edits on pattern images. By using a pattern analogy -- a pair of simple patterns to demonstrate the intended edit -- and a learning-based generative model to execute these edits, our method allows users to intuitively edit patterns. To enable this paradigm, we introduce SplitWeave, a domain-specific language that, combined with a framework for sampling synthetic pattern analogies, enables the creation of a large, high-quality synthetic training dataset. We also present TriFuser, a Latent Diffusion Model (LDM) designed to overcome critical issues that arise when naively deploying LDMs to this task. Extensive experiments on real-world, artist-sourced patterns reveals that our method faithfully performs the demonstrated edit while also generalizing to related pattern styles beyond its training distribution., Comment: Website: https://bardofcodes.github.io/patterns/

We investigate a quantum dynamical phase transition induced by the competition between local unitary evolution and dissipation in a qubit chain with a strong, on-site $\mathbb{Z}_2$ symmetry. While the steady-state of this evolution is always maximally-mixed, we show that the dynamical behavior of certain non-local observables on the approach to this steady-state is dictated by a quantum Ising model with a $\textit{complex}$ transverse-field (cTFIM). We investigate these observables analytically, uncovering a dynamical phase transition as the relative rate of unitary evolution and dissipation is tuned. We show that the weak-dissipation limit corresponds to a cTFIM with a large magnitude of the imaginary transverse-field, for which the many-body "ground-state" (with smallest real eigenvalue) is gapless, exhibiting quasi-long-range correlations of the local magnetization with a continuously-varying exponent. Correspondingly, the dynamics of the non-local observables show oscillatory behavior with an amplitude decaying exponentially in time. The strong-dissipation limit corresponds to a gapped ferromagnetic phase of the cTFIM, and non-local observables show exponential decay on the approach to equilibrium. This transition in (1+1)-dimensions has a peculiar, "two-sided" nature appearing as either first- or second-order depending on the phase from which the transition is approached, an analytic result which is corroborated by numerical studies. In higher dimensions, we present a field-theoretic understanding of the first-order nature of this transition, when approaching from the ferromagnetic phase of the cTFIM, though the nature of the phase with large imaginary transverse-field remains to be understood., Comment: 15 + 14 pages, 8 figures

We study in detail the postselection problem in a specific model: bosons hopping on a lattice subjected to continuous local measurements of quadrature observables. We solve the model analytically and show that the postselection overhead can be reduced by postprocessing the entire measurement record into one or two numbers for each trajectory and then postselecting based only on these numbers. We then provide a step-by-step protocol designed to recover connected two-point functions of the quantum trajectories, which display an exponentially decaying profile that is not observable in the unconditional, trajectory averaged, state. With the analytical solution in hand, we analyse the features of this postprocessing stage with the intention of abstracting away the properties that make postselection feasible in this model and may help in mitigating postselection in more general settings. We also test the protocol numerically in a way that utilizes only experimentally accessible information, showing that various quantum trajectory observables can be recovered with a few repetitions of the numerical experiment, even after including inevitable coarse-graining procedures expected under realistic experimental conditions. Furthermore, all the information required to design the postprocessing stage is independently present both in the unconditional dynamics and also in the measurement record, thus bypassing the need to solve for the conditional evolution of the model. We finalize by providing experimental implementations of these models in cavity-QED and circuit-QED., Comment: 16 + 8 pages, 11 figures

We demonstrate that, starting with a simple fermion wave function, the steady mixed state of the evolution of a class of Lindbladians, and the ensemble created by strong local measurement of fermion density without post-selection can be mapped to the "Gutzwiller projected" wave functions in the doubled Hilbert space -- the representation of the density matrix through the Choi-Jamiolkowski isomorphism. A Gutzwiller projection is a broadly used approach of constructing spin liquid states. For example, if one starts with a gapless free Dirac fermion pure quantum state, the constructed mixed state corresponds to an algebraic spin liquid in the doubled Hilbert space. We also predict that for some initial fermion wave function, the mixed state created following the procedure described above is expected to have a spontaneous "strong-to-weak" U(1) symmetry breaking, which corresponds to the emergence of superconductivity in the doubled Hilbert space. We also design the experimental protocol to construct the desired physics of mixed states., Comment: 11 pages, 2 figures, more results and new authors added

We introduce Temporal Residual Jacobians as a novel representation to enable data-driven motion transfer. Our approach does not assume access to any rigging or intermediate shape keyframes, produces geometrically and temporally consistent motions, and can be used to transfer long motion sequences. Central to our approach are two coupled neural networks that individually predict local geometric and temporal changes that are subsequently integrated, spatially and temporally, to produce the final animated meshes. The two networks are jointly trained, complement each other in producing spatial and temporal signals, and are supervised directly with 3D positional information. During inference, in the absence of keyframes, our method essentially solves a motion extrapolation problem. We test our setup on diverse meshes (synthetic and scanned shapes) to demonstrate its superiority in generating realistic and natural-looking animations on unseen body shapes against SoTA alternatives. Supplemental video and code are available at https://temporaljacobians.github.io/ ., Comment: 15 pages, 6 figures

Effective representation of 2D images is fundamental in digital image processing, where traditional methods like raster and vector graphics struggle with sharpness and textural complexity respectively. Current neural fields offer high-fidelity and resolution independence but require predefined meshes with known discontinuities, restricting their utility. We observe that by treating all mesh edges as potential discontinuities, we can represent the magnitude of discontinuities with continuous variables and optimize. Based on this observation, we introduce a novel discontinuous neural field model that jointly approximate the target image and recovers discontinuities. Through systematic evaluations, our neural field demonstrates superior performance in denoising and super-resolution tasks compared to InstantNGP, achieving improvements of over 5dB and 10dB, respectively. Our model also outperforms Mumford-Shah-based methods in accurately capturing discontinuities, with Chamfer distances 3.5x closer to the ground truth. Additionally, our approach shows remarkable capability in handling complex artistic drawings and natural images.

Measurements can detect errors in a decohered quantum memory allowing active error correction to increase the memory time. Previous understanding of this mechanism has focused on evaluating the performance of error correction algorithms based on measurement results. In this work, we instead intrinsically characterize the information dynamics in a quantum memory under repeated measurements, using coherent information and relative entropy. We consider the dynamics of a $d$-dimensional stabilizer code subject to Pauli errors and noisy stabilizer measurements and develop a $(d+1)$-dimensional statistical mechanics model for the information-theoretic diagnostics. Our model is dual to the model previously obtained for the optimal decoding algorithm, and the potential decoding transition in the quantum memory again manifests as a thermal phase transition in the statistical mechanics model. We explicitly derive the model and study the phase transition in information encoding in three examples: surface codes, repetition codes, and the XZZX code., Comment: 27 pages, 9 figures

The success of denoising diffusion models in representing rich data distributions over 2D raster images has prompted research on extending them to other data representations, such as vector graphics. Unfortunately due to their variable structure and scarcity of vector training data, directly applying diffusion models on this domain remains a challenging problem. Using workarounds like optimization via Score Distillation Sampling (SDS) is also fraught with difficulty, as vector representations are non trivial to directly optimize and tend to result in implausible geometries such as redundant or self-intersecting shapes. NIVeL addresses these challenges by reinterpreting the problem on an alternative, intermediate domain which preserves the desirable properties of vector graphics -- mainly sparsity of representation and resolution-independence. This alternative domain is based on neural implicit fields expressed in a set of decomposable, editable layers. Based on our experiments, NIVeL produces text-to-vector graphics results of significantly better quality than the state-of-the-art.

We present personalized residuals and localized attention-guided sampling for efficient concept-driven generation using text-to-image diffusion models. Our method first represents concepts by freezing the weights of a pretrained text-conditioned diffusion model and learning low-rank residuals for a small subset of the model's layers. The residual-based approach then directly enables application of our proposed sampling technique, which applies the learned residuals only in areas where the concept is localized via cross-attention and applies the original diffusion weights in all other regions. Localized sampling therefore combines the learned identity of the concept with the existing generative prior of the underlying diffusion model. We show that personalized residuals effectively capture the identity of a concept in ~3 minutes on a single GPU without the use of regularization images and with fewer parameters than previous models, and localized sampling allows using the original model as strong prior for large parts of the image., Comment: CVPR 2024. Project page at https://cusuh.github.io/personalized-residuals

Animated logos are a compelling and ubiquitous way individuals and brands represent themselves online. Manually authoring these logos can require significant artistic skill and effort. To help novice designers animate logos, design tools currently offer templates and animation presets. However, these solutions can be limited in their expressive range. Large language models have the potential to help novice designers create animated logos by generating animation code that is tailored to their content. In this paper, we introduce LogoMotion, an LLM-based system that takes in a layered document and generates animated logos through visually-grounded program synthesis. We introduce techniques to create an HTML representation of a canvas, identify primary and secondary elements, synthesize animation code, and visually debug animation errors. When compared with an industry standard tool, we find that LogoMotion produces animations that are more content-aware and are on par in terms of quality. We conclude with a discussion of the implications of LLM-generated animation for motion design.

Procedural noise is a fundamental component of computer graphics pipelines, offering a flexible way to generate textures that exhibit "natural" random variation. Many different types of noise exist, each produced by a separate algorithm. In this paper, we present a single generative model which can learn to generate multiple types of noise as well as blend between them. In addition, it is capable of producing spatially-varying noise blends despite not having access to such data for training. These features are enabled by training a denoising diffusion model using a novel combination of data augmentation and network conditioning techniques. Like procedural noise generators, the model's behavior is controllable via interpretable parameters and a source of randomness. We use our model to produce a variety of visually compelling noise textures. We also present an application of our model to improving inverse procedural material design; using our model in place of fixed-type noise nodes in a procedural material graph results in higher-fidelity material reconstructions without needing to know the type of noise in advance., Comment: In ACM Transactions on Graphics (Proceedings of SIGGRAPH) 2024, 21 pages

Quantum systems subject to random unitary evolution and measurements at random points in spacetime exhibit entanglement phase transitions which depend on the frequency of these measurements. Past work has experimentally observed entanglement phase transitions on near-term quantum computers, but the characterization approach using entanglement entropy is not scalable due to exponential overhead of quantum state tomography and post selection. Recently, an alternative protocol to detect entanglement phase transitions using linear cross-entropy was proposed which eliminates both bottlenecks. Here, we report the demonstration of this protocol in systems with one-dimensional and all-to-all connectivities on IBM's quantum hardware on up to 22 qubits, a regime which is presently inaccessible if post-selection is required. We demonstrate a collapse of the data into a scale-invariant form with critical exponents agreeing with theory within uncertainty. Our demonstration paves the way for studies of measurement-induced entanglement phase transitions and associated critical phenomena on larger near-term quantum systems., Comment: 23 pages, 15 figures

Current controls over diffusion models (e.g., through text or ControlNet) for image generation fall short in recognizing abstract, continuous attributes like illumination direction or non-rigid shape change. In this paper, we present an approach for allowing users of text-to-image models to have fine-grained control of several attributes in an image. We do this by engineering special sets of input tokens that can be transformed in a continuous manner -- we call them Continuous 3D Words. These attributes can, for example, be represented as sliders and applied jointly with text prompts for fine-grained control over image generation. Given only a single mesh and a rendering engine, we show that our approach can be adopted to provide continuous user control over several 3D-aware attributes, including time-of-day illumination, bird wing orientation, dollyzoom effect, and object poses. Our method is capable of conditioning image creation with multiple Continuous 3D Words and text descriptions simultaneously while adding no overhead to the generative process. Project Page: https://ttchengab.github.io/continuous_3d_words, Comment: Project Page: https://ttchengab.github.io/continuous_3d_words

How to frame (or crop) a photo often depends on the image subject and its context; e.g., a human portrait. Recent works have defined the subject-aware image cropping task as a nuanced and practical version of image cropping. We propose a weakly-supervised approach (GenCrop) to learn what makes a high-quality, subject-aware crop from professional stock images. Unlike supervised prior work, GenCrop requires no new manual annotations beyond the existing stock image collection. The key challenge in learning from this data, however, is that the images are already cropped and we do not know what regions were removed. Our insight is to combine a library of stock images with a modern, pre-trained text-to-image diffusion model. The stock image collection provides diversity and its images serve as pseudo-labels for a good crop, while the text-image diffusion model is used to out-paint (i.e., outward inpainting) realistic uncropped images. Using this procedure, we are able to automatically generate a large dataset of cropped-uncropped training pairs to train a cropping model. Despite being weakly-supervised, GenCrop is competitive with state-of-the-art supervised methods and significantly better than comparable weakly-supervised baselines on quantitative and qualitative evaluation metrics., Comment: AAAI 24. Extended version with supplemental materials

We present VecFusion, a new neural architecture that can generate vector fonts with varying topological structures and precise control point positions. Our approach is a cascaded diffusion model which consists of a raster diffusion model followed by a vector diffusion model. The raster model generates low-resolution, rasterized fonts with auxiliary control point information, capturing the global style and shape of the font, while the vector model synthesizes vector fonts conditioned on the low-resolution raster fonts from the first stage. To synthesize long and complex curves, our vector diffusion model uses a transformer architecture and a novel vector representation that enables the modeling of diverse vector geometry and the precise prediction of control points. Our experiments show that, in contrast to previous generative models for vector graphics, our new cascaded vector diffusion model generates higher quality vector fonts, with complex structures and diverse styles.

We introduce a novel image-to-painting method that facilitates the creation of large-scale, high-fidelity paintings with human-like quality and stylistic variation. To process large images and gain control over the painting process, we introduce a segmentation-based painting process and a dynamic attention map approach inspired by human painting strategies, allowing optimization of brush strokes to proceed in batches over different image regions, thereby capturing both large-scale structure and fine details, while also allowing stylistic control over detail. Our optimized batch processing and patch-based loss framework enable efficient handling of large canvases, ensuring our painted outputs are both aesthetically compelling and functionally superior as compared to previous methods, as confirmed by rigorous evaluations. Code available at: https://github.com/manuelladron/semantic\_based\_painting.git, Comment: 8 pages

Exploring variations of 3D shapes is a time-consuming process in traditional 3D modeling tools. Deep generative models of 3D shapes often feature continuous latent spaces that can, in principle, be used to explore potential variations starting from a set of input shapes. In practice, doing so can be problematic: latent spaces are high dimensional and hard to visualize, contain shapes that are not relevant to the input shapes, and linear paths through them often lead to sub-optimal shape transitions. Furthermore, one would ideally be able to explore variations in the original high-quality meshes used to train the generative model, not its lower-quality output geometry. In this paper, we present a method to explore variations among a given set of landmark shapes by constructing a mapping from an easily-navigable 2D exploration space to a subspace of a pre-trained generative model. We first describe how to find a mapping that spans the set of input landmark shapes and exhibits smooth variations between them. We then show how to turn the variations in this subspace into deformation fields, to transfer those variations to high-quality meshes for the landmark shapes. Our results show that our method can produce visually-pleasing and easily-navigable 2D exploration spaces for several different shape categories, especially as compared to prior work on learning deformation spaces for 3D shapes., Comment: SIGGRAPH Asia 2023, 15 pages

The linear cross-entropy (LXE) has been recently proposed as a scalable probe of the measurement-driven phase transition between volume- and area-law-entangled phases of pure-state trajectories in certain monitored quantum circuits. Here, we demonstrate that the LXE can distinguish distinct area-law-entangled phases of monitored circuits with symmetries, and extract universal behavior at the critical points separating these phases. We focus on (1+1)-dimensional monitored circuits with an on-site $\mathbb{Z}_{2}$ symmetry. For an appropriate choice of initial states, the LXE distinguishes the area-law-entangled spin glass and paramagnetic phases of the monitored trajectories. At the critical point, described by two-dimensional percolation, the LXE exhibits universal behavior which depends sensitively on boundary conditions, and the choice of initial states. With open boundary conditions, we show that the LXE relates to crossing probabilities in critical percolation, and is thus given by a known universal function of the aspect ratio of the dynamics, which quantitatively agrees with numerical studies of the LXE at criticality. The LXE probes correlations of other operators in percolation with periodic boundary conditions. We show that the LXE is sensitive to the richer phase diagram of the circuit model in the presence of symmmetric unitary gates. Lastly, we consider the effect of noise during the circuit evolution, and propose potential solutions to counter it., Comment: 12+6 pages, 16 figures. V2: References added

Automatic generation of fonts can be an important aid to typeface design. Many current approaches regard glyphs as pixelated images, which present artifacts when scaling and inevitable quality losses after vectorization. On the other hand, existing vector font synthesis methods either fail to represent the shape concisely or require vector supervision during training. To push the quality of vector font synthesis to the next level, we propose a novel dual-part representation for vector glyphs, where each glyph is modeled as a collection of closed "positive" and "negative" path pairs. The glyph contour is then obtained by boolean operations on these paths. We first learn such a representation only from glyph images and devise a subsequent contour refinement step to align the contour with an image representation to further enhance details. Our method, named DualVector, outperforms state-of-the-art methods in vector font synthesis both quantitatively and qualitatively. Our synthesized vector fonts can be easily converted to common digital font formats like TrueType Font for practical use. The code is released at https://github.com/thuliu-yt16/dualvector., Comment: CVPR 2023

Adaptive quantum circuits, in which unitary operations, measurements, and feedback are used to steer quantum many-body systems, provide an exciting opportunity to generate new dynamical steady states. We introduce an adaptive quantum dynamics with continuous symmetry where unitary operations, measurements, and local unitary feedback are used to drive ordering. In this setting, we find a pure steady state hosting symmetry-breaking order, which is the ground state of a gapless, local Hamiltonian. We explore the dynamical properties of the approach to this steady state. We find that this steady-state order is fragile to perturbations, even those that respect the continuous symmetry., Comment: 17 pages, 10 figures

This report surveys advances in deep learning-based modeling techniques that address four different 3D indoor scene analysis tasks, as well as synthesis of 3D indoor scenes. We describe different kinds of representations for indoor scenes, various indoor scene datasets available for research in the aforementioned areas, and discuss notable works employing machine learning models for such scene modeling tasks based on these representations. Specifically, we focus on the analysis and synthesis of 3D indoor scenes. With respect to analysis, we focus on four basic scene understanding tasks -- 3D object detection, 3D scene segmentation, 3D scene reconstruction and 3D scene similarity. And for synthesis, we mainly discuss neural scene synthesis works, though also highlighting model-driven methods that allow for human-centric, progressive scene synthesis. We identify the challenges involved in modeling scenes for these tasks and the kind of machinery that needs to be developed to adapt to the data representation, and the task setting in general. For each of these tasks, we provide a comprehensive summary of the state-of-the-art works across different axes such as the choice of data representation, backbone, evaluation metric, input, output, etc., providing an organized review of the literature. Towards the end, we discuss some interesting research directions that have the potential to make a direct impact on the way users interact and engage with these virtual scene models, making them an integral part of the metaverse., Comment: Published in Computer Graphics Forum, Aug 2023

Representing a 3D shape with a set of primitives can aid perception of structure, improve robotic object manipulation, and enable editing, stylization, and compression of 3D shapes. Existing methods either use simple parametric primitives or learn a generative shape space of parts. Both have limitations: parametric primitives lead to coarse approximations, while learned parts offer too little control over the decomposition. We instead propose to decompose shapes using a library of 3D parts provided by the user, giving full control over the choice of parts. The library can contain parts with high-quality geometry that are suitable for a given category, resulting in meaningful decompositions with clean geometry. The type of decomposition can also be controlled through the choice of parts in the library. Our method works via a self-supervised approach that iteratively retrieves parts from the library and refines their placements. We show that this approach gives higher reconstruction accuracy and more desirable decompositions than existing approaches. Additionally, we show how the decomposition can be controlled through the part library by using different part libraries to reconstruct the same shapes., Comment: CVPR 2023

The Floquet code utilizes a periodic sequence of two-qubit measurements to realize the topological order. After each measurement round, the instantaneous stabilizer group can be mapped to a honeycomb toric code, explaining the topological feature. The code also possesses a time-crystal order - the $e-m$ transmutation after every cycle, breaking the Floquet symmetry of the measurement schedule. This behavior is distinct from the stationary topological order realized in either random circuits or time-independent Hamiltonian. Therefore, the resultant phase belongs to the overlap between the classes of Floquet enriched topological orders and measurement-induced phases. In this work, we construct a continuous path interpolating between the Floquet and toric codes, focusing on the transition between the time-crystal and stationary topological phases. We show that this transition is characterized by a divergent length scale. We also add single-qubit perturbations to the model and obtain a richer two-dimensional parametric phase diagram of the Floquet code, showing the stability of the Floquet enriched topological order., Comment: 6+8 pages, 13 figures

Diffusion models currently achieve state-of-the-art performance for both conditional and unconditional image generation. However, so far, image diffusion models do not support tasks required for 3D understanding, such as view-consistent 3D generation or single-view object reconstruction. In this paper, we present RenderDiffusion, the first diffusion model for 3D generation and inference, trained using only monocular 2D supervision. Central to our method is a novel image denoising architecture that generates and renders an intermediate three-dimensional representation of a scene in each denoising step. This enforces a strong inductive structure within the diffusion process, providing a 3D consistent representation while only requiring 2D supervision. The resulting 3D representation can be rendered from any view. We evaluate RenderDiffusion on FFHQ, AFHQ, ShapeNet and CLEVR datasets, showing competitive performance for generation of 3D scenes and inference of 3D scenes from 2D images. Additionally, our diffusion-based approach allows us to use 2D inpainting to edit 3D scenes., Comment: Accepted at CVPR 2023. Project page: https://github.com/Anciukevicius/RenderDiffusion

The coherent superposition of quantum states is an important resource for quantum information processing which distinguishes quantum dynamics and information from their classical counterparts. In this article we determine the coherence requirements to communicate quantum information in a broad setting encompassing monitored quantum dynamics and quantum error correction codes. We determine these requirements by considering hybrid circuits that are generated by a quantum information game played between two opponents, Alice and Eve, who compete by applying unitaries and measurements on a fixed number of qubits. Alice applies unitaries in an attempt to maintain quantum channel capacity, while Eve applies measurements in an attempt to destroy it. By limiting the coherence generating or destroying operations available to each opponent, we determine Alice's coherence requirements. When Alice plays a random strategy aimed at mimicking generic monitored quantum dynamics, we discover a coherence-tuned phase transitions in entanglement and quantum channel capacity. We then derive a theorem giving the minimum coherence required by Alice in any successful strategy, and conclude by proving that coherence sets an upper bound on the code distance in any stabelizer quantum error correction codes. Such bounds provide a rigorous quantification of the coherence resource requirements for quantum communication and error correction., Comment: 19+11 pages, 12+3 figures

We investigate the prospects of employing the linear cross-entropy to experimentally access measurement-induced phase transitions (MIPT) without requiring any postselection of quantum trajectories. For two random circuits that are identical in the bulk but with different initial states, the linear cross-entropy $\chi$ between the bulk measurement outcome distributions in the two circuits acts as a boundary order parameter, and can be used to distinguish the volume law from area law phases. In the volume law phase (and in the thermodynamic limit) the bulk measurements cannot distinguish between the two different initial states, and $\chi = 1$. In the area law phase $\chi < 1$. For circuits with Clifford gates, we provide numerical evidence that $\chi$ can be sampled to accuracy $\epsilon$ from $O(1/\epsilon^2)$ trajectories, by running the first circuit on a quantum simulator without postselection, aided by a classical simulation of the second. We also find that for weak depolarizing noise the signature of the MIPT is still present for intermediate system sizes. In our protocol we have the freedom of choosing initial states such that the "classical" side can be simulated efficiently, while simulating the "quantum" side is still classically hard., Comment: 7+8 pages, 6 figures. v2: 7+9 pages, 3+3 figures. Updated discussions on sample size (Fig. 2d, 2e), and new results from random Haar circuits (Fig. 3b). Accepted version

Measurements allow efficient preparation of interesting quantum many-body states with long-range entanglement, conditioned on additional transformations based on measurement outcomes. Here, we demonstrate that the so-called conformal quantum critical points (CQCP) can be obtained by performing general single-site measurements in an appropriate basis on the cluster states in $d\geq2$. The equal-time correlators of the said states are described by correlation functions of certain $d$-dimensional classical models at finite temperatures and feature spatial conformal invariance. This establishes an exact correspondence between the measurement-prepared critical states and conformal field theories of a range of critical spin models, including familiar Ising models and gauge theories. Furthermore, by mapping the long-range entanglement structure of measured quantum states into the correlations of the corresponding thermal spin model, we rigorously establish the stability condition of the long-range entanglement in the measurement-prepared quantum states deviating from the ideal setting. Most importantly, we describe protocols to decode the resulting quantum phases and transitions without post-selection, thus transferring the exponential measurement complexity to a polynomial classical computation. Therefore, our findings suggest a novel mechanism in which a quantum critical wavefunction emerges, providing new practical ways to study quantum phases and conformal quantum critical points., Comment: 37 pages, 11 figures

Quantum circuits -- built from local unitary gates and local measurements -- are a new playground for quantum many-body physics and a tractable setting to explore universal collective phenomena far-from-equilibrium. These models have shed light on longstanding questions about thermalization and chaos, and on the underlying universal dynamics of quantum information and entanglement. In addition, such models generate new sets of questions and give rise to phenomena with no traditional analog, such as new dynamical phases in quantum systems that are monitored by an external observer. Quantum circuit dynamics is also topical in view of experimental progress in building digital quantum simulators that allow control of precisely these ingredients. Randomness in the circuit elements allows a high level of theoretical control, with a key theme being mappings between real-time quantum dynamics and effective classical lattice models or dynamical processes. Many of the universal phenomena that can be identified in this tractable setting apply to much wider classes of more structured many-body dynamics., Comment: Review article for Annual Review of Condensed Matter Physics; comments welcome

We introduce the task of spotting temporally precise, fine-grained events in video (detecting the precise moment in time events occur). Precise spotting requires models to reason globally about the full-time scale of actions and locally to identify subtle frame-to-frame appearance and motion differences that identify events during these actions. Surprisingly, we find that top performing solutions to prior video understanding tasks such as action detection and segmentation do not simultaneously meet both requirements. In response, we propose E2E-Spot, a compact, end-to-end model that performs well on the precise spotting task and can be trained quickly on a single GPU. We demonstrate that E2E-Spot significantly outperforms recent baselines adapted from the video action detection, segmentation, and spotting literature to the precise spotting task. Finally, we contribute new annotations and splits to several fine-grained sports action datasets to make these datasets suitable for future work on precise spotting., Comment: ECCV 2022; Website URL: https://jhong93.github.io/projects/spot.html

Stein discrepancies have emerged as a powerful statistical tool, being applied to fundamental statistical problems including parameter inference, goodness-of-fit testing, and sampling. The canonical Stein discrepancies require the derivatives of a statistical model to be computed, and in return provide theoretical guarantees of convergence detection and control. However, for complex statistical models, the stable numerical computation of derivatives can require bespoke algorithmic development and render Stein discrepancies impractical. This paper focuses on posterior approximation using Stein discrepancies, and introduces a collection of non-canonical Stein discrepancies that are gradient free, meaning that derivatives of the statistical model are not required. Sufficient conditions for convergence detection and control are established, and applications to sampling and variational inference are presented.

Rigged puppets are one of the most prevalent representations to create 2D character animations. Creating these puppets requires partitioning characters into independently moving parts. In this work, we present a method to automatically identify such articulated parts from a small set of character poses shown in a sprite sheet, which is an illustration of the character that artists often draw before puppet creation. Our method is trained to infer articulated parts, e.g. head, torso and limbs, that can be re-assembled to best reconstruct the given poses. Our results demonstrate significantly better performance than alternatives qualitatively and quantitatively.Our project page https://zhan-xu.github.io/parts/ includes our code and data.

We present ASSET, a neural architecture for automatically modifying an input high-resolution image according to a user's edits on its semantic segmentation map. Our architecture is based on a transformer with a novel attention mechanism. Our key idea is to sparsify the transformer's attention matrix at high resolutions, guided by dense attention extracted at lower image resolutions. While previous attention mechanisms are computationally too expensive for handling high-resolution images or are overly constrained within specific image regions hampering long-range interactions, our novel attention mechanism is both computationally efficient and effective. Our sparsified attention mechanism is able to capture long-range interactions and context, leading to synthesizing interesting phenomena in scenes, such as reflections of landscapes onto water or flora consistent with the rest of the landscape, that were not possible to generate reliably with previous convnets and transformer approaches. We present qualitative and quantitative results, along with user studies, demonstrating the effectiveness of our method., Comment: SIGGRAPH 2022 - Journal Track

Approaches for single-view reconstruction typically rely on viewpoint annotations, silhouettes, the absence of background, multiple views of the same instance, a template shape, or symmetry. We avoid all such supervision and assumptions by explicitly leveraging the consistency between images of different object instances. As a result, our method can learn from large collections of unlabelled images depicting the same object category. Our main contributions are two ways for leveraging cross-instance consistency: (i) progressive conditioning, a training strategy to gradually specialize the model from category to instances in a curriculum learning fashion; and (ii) neighbor reconstruction, a loss enforcing consistency between instances having similar shape or texture. Also critical to the success of our method are: our structured autoencoding architecture decomposing an image into explicit shape, texture, pose, and background; an adapted formulation of differential rendering; and a new optimization scheme alternating between 3D and pose learning. We compare our approach, UNICORN, both on the diverse synthetic ShapeNet dataset - the classical benchmark for methods requiring multiple views as supervision - and on standard real-image benchmarks (Pascal3D+ Car, CUB) for which most methods require known templates and silhouette annotations. We also showcase applicability to more challenging real-world collections (CompCars, LSUN), where silhouettes are not available and images are not cropped around the object., Comment: ECCV 2022. Project webpage with code and videos: http://imagine.enpc.fr/~monniert/UNICORN/

Sequential algorithms are popular for experimental design, enabling emulation, optimisation and inference to be efficiently performed. For most of these applications bespoke software has been developed, but the approach is general and many of the actual computations performed in such software are identical. Motivated by the diverse problems that can in principle be solved with common code, this paper presents GaussED, a simple probabilistic programming language coupled to a powerful experimental design engine, which together automate sequential experimental design for approximating a (possibly nonlinear) quantity of interest in Gaussian processes models. Using a handful of commands, GaussED can be used to: solve linear partial differential equations, perform tomographic reconstruction from integral data and implement Bayesian optimisation with gradient data.

This paper introduces a model for producing stylized line drawings from 3D shapes. The model takes a 3D shape and a viewpoint as input, and outputs a drawing with textured strokes, with variations in stroke thickness, deformation, and color learned from an artist's style. The model is fully differentiable. We train its parameters from a single training drawing of another 3D shape. We show that, in contrast to previous image-based methods, the use of a geometric representation of 3D shape and 2D strokes allows the model to transfer important aspects of shape and texture style while preserving contours. Our method outputs the resulting drawing in a vector representation, enabling richer downstream analysis or editing in interactive applications., Comment: Accepted to ICCV 2021

We introduce an exact mapping of Clifford (stabilizer) random tensor networks (RTNs) and monitored quantum circuits, onto a statistical mechanics model. With Haar unitaries, the fundamental degrees of freedom ('spins') are permutations because all operators commuting with the action of the unitaries on a tensor product arise from permutations of the tensor factors ('Schur-Weyl duality'). For unitaries restricted to the smaller Clifford group, the set of commuting operators, the 'commutant', forming the new 'spin' degrees of freedom, will be larger. We use the recent full characterization of this commutant by Gross et al., Comm. Math. Phys. 385, 1325 (2021), to construct the Clifford statistical mechanics models for on-site Hilbert space dimensions which are powers of a prime number $p$. We show that the Boltzmann weights are invariant under a symmetry group involving orthogonal matrices with entries in the finite number field ${\bf F}_p$. This implies that the symmetry group, and consequently all universal properties of entanglement transitions in Clifford circuits and RTNs will in general depend on, and only on the prime $p$. We show that Clifford monitored circuits with on-site Hilbert space dimension $d=p^M$ are described by percolation in the limits $d \to \infty$ at (a) $p=$ fixed but $M\to \infty$, and at (b) $M= 1$ but $p \to \infty$. In the limit (a) we calculate the effective central charge, and in the limit (b) we derive the following universal minimal cut entanglement entropy $S_A =(\sqrt{3}/\pi)\ln p \ln L_A$ for $d=p$ large at the transition. We verify those predictions numerically, and present extensive numerical results for critical exponents at the transition in monitored Clifford circuits for prime number on-site Hilbert space dimension $d=p$ for a variety of different values of $p$, and find that they approach percolation values at large $p$., Comment: 23 pages, 5 figures. Abstract shortened to meet arxiv requirements, see pdf for full abstract

Human pose is a useful feature for fine-grained sports action understanding. However, pose estimators are often unreliable when run on sports video due to domain shift and factors such as motion blur and occlusions. This leads to poor accuracy when downstream tasks, such as action recognition, depend on pose. End-to-end learning circumvents pose, but requires more labels to generalize. We introduce Video Pose Distillation (VPD), a weakly-supervised technique to learn features for new video domains, such as individual sports that challenge pose estimation. Under VPD, a student network learns to extract robust pose features from RGB frames in the sports video, such that, whenever pose is considered reliable, the features match the output of a pretrained teacher pose detector. Our strategy retains the best of both pose and end-to-end worlds, exploiting the rich visual patterns in raw video frames, while learning features that agree with the athletes' pose and motion in the target video domain to avoid over-fitting to patterns unrelated to athletes' motion. VPD features improve performance on few-shot, fine-grained action recognition, retrieval, and detection tasks in four real-world sports video datasets, without requiring additional ground-truth pose annotations., Comment: ICCV 2021 (poster)