Search

We show that a class of $L$-loop conformal ladder graphs correspond to twisted partition functions of free massive complex scalars in $d=2L+1$ dimensions. The graphs arise as four-point functions in certain two- and four-dimensional conformal fishnet models. The twisted thermal two-point function of the scalars is a generator of such conformal graphs for all loops. We argue that this correspondence is seeded by a system of two decoupled harmonic oscillators twisted by an imaginary chemical potential. We find a number of algebraic and differential relations among the conformal graphs which mirror the underlying free dynamics., Comment: V1: LaTeX 6 pages, double column, 2 figures V2: LaTex 7 pages, two appendices with technical details added, few typos corrected. Matched published version

Formulated as a conditional generation problem, face animation aims at synthesizing continuous face images from a single source image driven by a set of conditional face motion. Previous works mainly model the face motion as conditions with 1D or 2D representation (e.g., action units, emotion codes, landmark), which often leads to low-quality results in some complicated scenarios such as continuous generation and largepose transformation. To tackle this problem, the conditions are supposed to meet two requirements, i.e., motion information preserving and geometric continuity. To this end, we propose a novel representation based on a 3D geometric flow, termed facial flow, to represent the natural motion of the human face at any pose. Compared with other previous conditions, the proposed facial flow well controls the continuous changes to the face. After that, in order to utilize the facial flow for face editing, we build a synthesis framework generating continuous images with conditional facial flows. To fully take advantage of the motion information of facial flows, a hierarchical conditional framework is designed to combine the extracted multi-scale appearance features from images and motion features from flows in a hierarchical manner. The framework then decodes multiple fused features back to images progressively. Experimental results demonstrate the effectiveness of our method compared to other state-of-the-art methods.

High-capacity image steganography, aimed at concealing a secret image in a cover image, is a technique to preserve sensitive data, e.g., faces and fingerprints. Previous methods focus on the security during transmission and subsequently run a risk of privacy leakage after the restoration of secret images at the receiving end. To address this issue, we propose a framework, called Multitask Identity-Aware Image Steganography (MIAIS), to achieve direct recognition on container images without restoring secret images. The key issue of the direct recognition is to preserve identity information of secret images into container images and make container images look similar to cover images at the same time. Thus, we introduce a simple content loss to preserve the identity information, and design a minimax optimization to deal with the contradictory aspects. We demonstrate that the robustness results can be transferred across different cover datasets. In order to be flexible for the secret image restoration in some cases, we incorporate an optional restoration network into our method, providing a multitask framework. The experiments under the multitask scenario show the effectiveness of our framework compared with other visual information hiding methods and state-of-the-art high-capacity image steganography methods., Comment: Accepted to Transaction of Image Processing

We study $AdS_3 \times S^1 \times Y$ supersymmetric string theory backgrounds with Neveu-Schwarz-Neveu-Schwarz flux that are dual to ${\cal N}=2$ superconformal theories on the boundary. We classify all worldsheet vertex operators that correspond to space-time chiral primaries. We compute space-time chiral ring structure constants for operators in the zero spectral flow sector using the operator product expansion in the worldsheet theory. We find that the structure constants take a universal form that depends only on the topological data of the ${\cal N}=2$ superconformal theory on $Y$., Comment: 44 pages

Gait recognition has a rapid development in recent years. However, gait recognition in the wild is not well explored yet. An obvious reason could be ascribed to the lack of diverse training data from the perspective of intrinsic and extrinsic factors. To remedy this problem, we propose to construct a large-scale gait dataset with the help of controllable computer simulation. In detail, to diversify the intrinsic factors of gait, we generate numerous characters with diverse attributes and empower them with various types of walking styles. To diversify the extrinsic factors of gait, we build a complicated scene with a dense camera layout. Finally, we design an automated generation toolkit under Unity3D for simulating the walking scenario and capturing the gait data automatically. As a result, we obtain an in-the-wild gait dataset, called VersatileGait, which has more than one million silhouette sequences of 10,000 subjects with diverse scenarios. VersatileGait possesses several nice properties, including huge dataset size, diverse pedestrian attributes, complicated camera layout, high-quality annotations, small domain gap with the real one, good scalability for new demands, and no privacy issues. Based on VersatileGait, we propose series of experiments and applications for both research exploration of gait in the wild and practical applications. Our dataset and its corresponding generation toolkit will be publicly available for further studies., Comment: We should have updated 2101.01394 but we did a new submission

Deep Learning has implemented a wide range of applications and has become increasingly popular in recent years. The goal of multimodal deep learning is to create models that can process and link information using various modalities. Despite the extensive development made for unimodal learning, it still cannot cover all the aspects of human learning. Multimodal learning helps to understand and analyze better when various senses are engaged in the processing of information. This paper focuses on multiple types of modalities, i.e., image, video, text, audio, body gestures, facial expressions, and physiological signals. Detailed analysis of past and current baseline approaches and an in-depth study of recent advancements in multimodal deep learning applications has been provided. A fine-grained taxonomy of various multimodal deep learning applications is proposed, elaborating on different applications in more depth. Architectures and datasets used in these applications are also discussed, along with their evaluation metrics. Last, main issues are highlighted separately for each domain along with their possible future research directions.

RGB-D salient object detection (SOD) is usually formulated as a problem of classification or regression over two modalities, i.e., RGB and depth. Hence, effective RGBD feature modeling and multi-modal feature fusion both play a vital role in RGB-D SOD. In this paper, we propose a depth-sensitive RGB feature modeling scheme using the depth-wise geometric prior of salient objects. In principle, the feature modeling scheme is carried out in a depth-sensitive attention module, which leads to the RGB feature enhancement as well as the background distraction reduction by capturing the depth geometry prior. Moreover, to perform effective multi-modal feature fusion, we further present an automatic architecture search approach for RGB-D SOD, which does well in finding out a feasible architecture from our specially designed multi-modal multi-scale search space. Extensive experiments on seven standard benchmarks demonstrate the effectiveness of the proposed approach against the state-of-the-art., Comment: Accepted by CVPR2021, Oral

Unsupervised domain adaptation (UDA) typically carries out knowledge transfer from a label-rich source domain to an unlabeled target domain by adversarial learning. In principle, existing UDA approaches mainly focus on the global distribution alignment between domains while ignoring the intrinsic local distribution properties. Motivated by this observation, we propose an end-to-end structure-conditioned adversarial learning scheme (SCAL) that is able to preserve the intra-class compactness during domain distribution alignment. By using local structures as structure-aware conditions, the proposed scheme is implemented in a structure-conditioned adversarial learning pipeline. The above learning procedure is iteratively performed by alternating between local structures establishment and structure-conditioned adversarial learning. Experimental results demonstrate the effectiveness of the proposed scheme in UDA scenarios.

With the motivation of practical gait recognition applications, we propose to automatically create a large-scale synthetic gait dataset (called VersatileGait) by a game engine, which consists of around one million silhouette sequences of 11,000 subjects with fine-grained attributes in various complicated scenarios. Compared with existing real gait datasets with limited samples and simple scenarios, the proposed VersatileGait dataset possesses several nice properties, including huge dataset size, high sample diversity, high-quality annotations, multi-pitch angles, small domain gap with the real one, etc. Furthermore, we investigate the effectiveness of our dataset (e.g., domain transfer after pretraining). Then, we use the fine-grained attributes from VersatileGait to promote gait recognition in both accuracy and speed, and meanwhile justify the gait recognition performance under multi-pitch angle settings. Additionally, we explore a variety of potential applications for research.Extensive experiments demonstrate the value and effective-ness of the proposed VersatileGait in gait recognition along with its associated applications. We will release both VersatileGait and its corresponding data generation toolkit for further studies.

We analyse topological orbifold conformal field theories on the symmetric product of a complex surface M. By exploiting the mathematics literature we show that a canonical quotient of the operator ring has structure constants given by Hurwitz numbers. This proves a conjecture in the physics literature on extremal correlators. Moreover, it allows to leverage results on the combinatorics of the symmetric group to compute more structure constants explicitly. We recall that the full orbifold chiral ring is given by a symmetric orbifold Frobenius algebra. This construction enables the computation of topological genus zero and genus one correlators, and to prove the vanishing of higher genus contributions. The efficient description of all topological correlators sets the stage for a proof of a topological AdS/CFT correspondence. Indeed, we propose a concrete mathematical incarnation of the proof, relating Gromow-Witten theory in the bulk to the quantum cohomology of the Hilbert scheme on the boundary., Comment: 33 pages. v2: Remarks on proof added

Knowledge distillation, aimed at transferring the knowledge from a heavy teacher network to a lightweight student network, has emerged as a promising technique for compressing neural networks. However, due to the capacity gap between the heavy teacher and the lightweight student, there still exists a significant performance gap between them. In this paper, we see knowledge distillation in a fresh light, using the knowledge gap, or the residual, between a teacher and a student as guidance to train a much more lightweight student, called a res-student. We combine the student and the res-student into a new student, where the res-student rectifies the errors of the former student. Such a residual-guided process can be repeated until the user strikes the balance between accuracy and cost. At inference time, we propose a sample-adaptive strategy to decide which res-students are not necessary for each sample, which can save computational cost. Experimental results show that we achieve competitive performance with 18.04$\%$, 23.14$\%$, 53.59$\%$, and 56.86$\%$ of the teachers' computational costs on the CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet datasets. Finally, we do thorough theoretical and empirical analysis for our method., Comment: The first two authors (Xuewei Li and Songyuan Li) contribute equally. Accepted to IEEE TRANSACTIONS ON IMAGE PROCESSING (TIP)

Dynamic routing networks, aimed at finding the best routing paths in the networks, have achieved significant improvements to neural networks in terms of accuracy and efficiency. In this paper, we see dynamic routing networks in a fresh light, formulating a routing method as a mapping from a sample space to a routing space. From the perspective of space mapping, prevalent methods of dynamic routing didn't consider how inference paths would be distributed in the routing space. Thus, we propose a novel method, termed CoDiNet, to model the relationship between a sample space and a routing space by regularizing the distribution of routing paths with the properties of consistency and diversity. Specifically, samples with similar semantics should be mapped into the same area in routing space, while those with dissimilar semantics should be mapped into different areas. Moreover, we design a customizable dynamic routing module, which can strike a balance between accuracy and efficiency. When deployed upon ResNet models, our method achieves higher performance and effectively reduces average computational cost on four widely used datasets.

We construct a string theory in three-dimensional anti-de Sitter space-time that is independent of the boundary metric. It is a topologically twisted theory of quantum gravity. We study string theories with an asymptotic N=2 superconformal symmetry and demonstrate that, when the world sheet coupling to the space-time boundary metric undergoes a U(1) R-symmetry twist, the space-time boundary energy-momentum tensor becomes topological. As a by-product of our analysis, we obtain the world sheet vertex operator that codes the space-time energy-momentum for conformally flat boundary metrics., Comment: 23 pages

Recent approaches for fast semantic video segmentation have reduced redundancy by warping feature maps across adjacent frames, greatly speeding up the inference phase. However, the accuracy drops seriously owing to the errors incurred by warping. In this paper, we propose a novel framework and design a simple and effective correction stage after warping. Specifically, we build a non-key-frame CNN, fusing warped context features with current spatial details. Based on the feature fusion, our Context Feature Rectification~(CFR) module learns the model's difference from a per-frame model to correct the warped features. Furthermore, our Residual-Guided Attention~(RGA) module utilizes the residual maps in the compressed domain to help CRF focus on error-prone regions. Results on Cityscapes show that the accuracy significantly increases from $67.3\%$ to $71.6\%$, and the speed edges down from $65.5$ FPS to $61.8$ FPS at a resolution of $1024\times 2048$. For non-rigid categories, e.g., ``human'' and ``object'', the improvements are even higher than 18 percentage points.

To satisfy the stringent requirements on computational resources in the field of real-time semantic segmentation, most approaches focus on the hand-crafted design of light-weight segmentation networks. Recently, Neural Architecture Search (NAS) has been used to search for the optimal building blocks of networks automatically, but the network depth, downsampling strategy, and feature aggregation way are still set in advance by trial and error. In this paper, we propose a joint search framework, called AutoRTNet, to automate the design of these strategies. Specifically, we propose hyper-cells to jointly decide the network depth and downsampling strategy, and an aggregation cell to achieve automatic multi-scale feature aggregation. Experimental results show that AutoRTNet achieves 73.9% mIoU on the Cityscapes test set and 110.0 FPS on an NVIDIA TitanXP GPU card with 768x1536 input images., Comment: submitted to IJCV

Panoptic segmentation aims to perform instance segmentation for foreground instances and semantic segmentation for background stuff simultaneously. The typical top-down pipeline concentrates on two key issues: 1) how to effectively model the intrinsic interaction between semantic segmentation and instance segmentation, and 2) how to properly handle occlusion for panoptic segmentation. Intuitively, the complementarity between semantic segmentation and instance segmentation can be leveraged to improve the performance. Besides, we notice that using detection/mask scores is insufficient for resolving the occlusion problem. Motivated by these observations, we propose a novel deep panoptic segmentation scheme based on a bidirectional learning pipeline. Moreover, we introduce a plug-and-play occlusion handling algorithm to deal with the occlusion between different object instances. The experimental results on COCO panoptic benchmark validate the effectiveness of our proposed method. Codes will be released soon at https://github.com/Mooonside/BANet., Comment: to be published in CVPR2020, oral paper

Real-time semantic video segmentation is a challenging task due to the strict requirements of inference speed. Recent approaches mainly devote great efforts to reducing the model size for high efficiency. In this paper, we rethink this problem from a different viewpoint: using knowledge contained in compressed videos. We propose a simple and effective framework, dubbed TapLab, to tap into resources from the compressed domain. Specifically, we design a fast feature warping module using motion vectors for acceleration. To reduce the noise introduced by motion vectors, we design a residual-guided correction module and a residual-guided frame selection module using residuals. TapLab significantly reduces redundant computations of the state-of-the-art fast semantic image segmentation models, running 3 to 10 times faster with controllable accuracy degradation. The experimental results show that TapLab achieves 70.6% mIoU on the Cityscapes dataset at 99.8 FPS with a single GPU card for the 1024x2048 videos. A high-speed version even reaches the speed of 160+ FPS. Codes will be available soon at https://github.com/Sixkplus/TapLab., Comment: Accepted to TPAMI

We analyse a simple example of a holographically dual pair in which we topologically twist both theories. The holography is based on the two-dimensional N=2 supersymmetric Liouville conformal field theory that defines a unitary bulk quantum supergravity theory in three-dimensional anti-de Sitter space. The supersymmetric version of three-dimensional Liouville quantum gravity allows for a topological twist on the boundary and in the bulk. We define the topological bulk supergravity theory in terms of twisted boundary conditions. We corroborate the duality by calculating the chiral configurations in the bulk supergravity theory and by quantising the solution space. Moreover, we note that the boundary calculation of the structure constants of the chiral ring carries over to the bulk theory as well. We thus construct a topological AdS/CFT duality in which the bulk theory is independent of the boundary metric., Comment: 21 pages

The increasing momentum of service-oriented architecture has led to the emergence of divergent delivered services, where service selection is meritedly required to obtain the target service fulfilling the requirements from both users and service providers. Despite many existing works have extensively handled the issue of service selection, it remains an open question in the case where requests from multiple users are performed simultaneously by a certain set of shared candidate services. Meanwhile, there exist some constraints enforced on the context of service selection, e.g. service placement location and contracts between users and service providers. In this paper, we focus on the QoS-aware service selection with constraints from a fairness aspect, with the objective of achieving max-min fairness across multiple service requests sharing candidate service sets. To be more specific, we study the problem of fairly selecting services from shared candidate sets while service providers are self-motivated to offer better services with higher QoS values. We formulate this problem as a lexicographical maximization problem, which is far from trivial to deal with practically due to its inherently multi-objective and discrete nature. A fairness-aware algorithm for concurrent service selection (FASS) is proposed, whose basic idea is to iteratively solve the single-objective subproblems by transforming them into linear programming problems. Experimental results based on real-world datasets also validate the effectiveness and practicality of our proposed approach., Comment: IEEE International Conference on Web Services (IEEE ICWS 2019), 9 pages

We define a three-dimensional quantum theory of gravity as the holographic dual of the Liouville conformal field theory. The theory is consistent and unitary by definition. The corresponding theory of gravity with negative cosmological constant has peculiar properties. The quantum theory has no normalisable AdS3 vacuum. The model contains primary black holes with zero spin. All states can be interpreted as black holes dressed with boundary gravitons. There is a unique universal interaction between these states consistent with unitarity and the conformal symmetry of the model. This theory of gravity, though conceptually isolated from other models of quantum gravity, is worth scrutinising., Comment: 26 pages, journal version

We couple twisted non-compact N=(2,2) supersymmetric models to topological gravity in two dimensions. We propose expressions for the genus zero correlation functions based on a Kadomtsev-Petviashvili integrable hierarchy. Moreover, we prove recursion relations satisfied by the topological gravity amplitudes at all genera and compute characteristic critical exponents. We discuss the extent to which moving beyond the N=2 central charge barrier opens a window on two-dimensional gravity with central charge larger than one., Comment: 25 pages

We study interacting massive N=(2,2) supersymmetric field theories in two dimensions which arise from deforming conformal field theories with a continuous spectrum. Firstly, we deform N=2 superconformal Liouville theory with relevant operators, and twist the theory into a topological quantum field theory. These theories can be thought of as twisted Landau-Ginzburg models with negative power superpotential. We determine the structure constants of the chiral ring and therefore all correlators of these topological quantum field theories. We provide general formulas for the deformed operators of given charge as well as explicit solutions to the WDVV equations. Secondly, we analyze the topological anti-topological sector of the theory. We compute the metric at the conformal point through localization. Moreover, we show that the topological-anti-topological fusion differential equations on the metric in a family of non-compact theories takes the affine Toda form. The metric as a function of the family of theories is identical to the metric in certain deformed compact models. For a negative cubic power Landau-Ginzburg superpotential, for instance, it is governed by the Painlev\'e III differential equation., Comment: 32 pages