Search

Question answering over knowledge bases is considered a difficult problem due to the challenge of generalizing to a wide variety of possible natural language questions. Additionally, the heterogeneity of knowledge base schema items between different knowledge bases often necessitates specialized training for different knowledge base question-answering (KBQA) datasets. To handle questions over diverse KBQA datasets with a unified training-free framework, we propose KB-BINDER, which for the first time enables few-shot in-context learning over KBQA tasks. Firstly, KB-BINDER leverages large language models like Codex to generate logical forms as the draft for a specific question by imitating a few demonstrations. Secondly, KB-BINDER grounds on the knowledge base to bind the generated draft to an executable one with BM25 score matching. The experimental results on four public heterogeneous KBQA datasets show that KB-BINDER can achieve a strong performance with only a few in-context demonstrations. Especially on GraphQA and 3-hop MetaQA, KB-BINDER can even outperform the state-of-the-art trained models. On GrailQA and WebQSP, our model is also on par with other fully-trained models. We believe KB-BINDER can serve as an important baseline for future research. Our code is available at https://github.com/ltl3A87/KB-BINDER., Comment: Accepted to ACL 2023

The one-shot talking-head synthesis task aims to animate a source image to another pose and expression, which is dictated by a driving frame. Recent methods rely on warping the appearance feature extracted from the source, by using motion fields estimated from the sparse keypoints, that are learned in an unsupervised manner. Due to their lightweight formulation, they are suitable for video conferencing with reduced bandwidth. However, based on our study, current methods suffer from two major limitations: 1) unsatisfactory generation quality in the case of large head poses and the existence of observable pose misalignment between the source and the first frame in driving videos. 2) fail to capture fine yet critical face motion details due to the lack of semantic understanding and appropriate face geometry regularization. To address these shortcomings, we propose a novel method that leverages the rich face prior information, the proposed model can generate face videos with improved semantic consistency (improve baseline by $7\%$ in average keypoint distance) and expression-preserving (outperform baseline by $15 \%$ in average emotion embedding distance) under equivalent bandwidth. Additionally, incorporating such prior information provides us with a convenient interface to achieve highly controllable generation in terms of both pose and expression.

In this paper, we present an extended dissipaton equation of motion for studying the dynamics of electronic impurity systems. Compared with the original theoretical formalism, the quadratic couplings are introduced into the Hamiltonian accounting for the interaction between the impurity and its surrounding environment. By exploiting the quadratic fermionic dissipaton algebra, the proposed extended dissipaton equation of motion offers a powerful tool for studying the dynamical behaviors of electronic impurity systems, particularly in situations where nonequilibrium and strongly correlated effects play significant roles. Numerical demonstrations are carried out to investigate the temperature dependence of the Kondo resonance in the Kondo impurity model., Comment: 6 pages, 1 figure

We tackle the long video generation problem, i.e.~generating videos beyond the output length of video generation models. Due to the computation resource constraints, video generation models can only generate video clips that are relatively short compared with the length of real videos. Existing works apply a sliding window approach to generate long videos at inference time, which is often limited to generating recurrent events or homogeneous content. To generate long videos covering diverse content and multiple events, we propose to use additional guidance to control the video generation process. We further present a two-stage approach to the problem, which allows us to utilize existing video generation models to generate high-quality videos within a small time window while modeling the video holistically based on the input guidance. The proposed approach is complementary to existing efforts on video generation, which focus on generating realistic video within a fixed time window. Extensive experiments on challenging real-world videos validate the benefit of the proposed method, which improves over state-of-the-art by up to 9.5% in objective metrics and is preferred by users more than 80% of time.

Many applications can benefit from personalized image generation models, including image enhancement, video conferences, just to name a few. Existing works achieved personalization by fine-tuning one model for each person. While being successful, this approach incurs additional computation and storage overhead for each new identity. Furthermore, it usually expects tens or hundreds of examples per identity to achieve the best performance. To overcome these challenges, we propose an encoder-based approach for personalization. We learn an identity encoder which can extract an identity representation from a set of reference images of a subject, together with a diffusion generator that can generate new images of the subject conditioned on the identity representation. Once being trained, the model can be used to generate images of arbitrary identities given a few examples even if the model hasn't been trained on the identity. Our approach greatly reduces the overhead for personalized image generation and is more applicable in many potential applications. Empirical results show that our approach consistently outperforms existing fine-tuning based approach in both image generation and reconstruction, and the outputs is preferred by users more than 95% of the time compared with the best performing baseline.

Knowledge tracing (KT) aims to assess individuals' evolving knowledge states according to their learning interactions with different exercises in online learning systems (OIS), which is critical in supporting decision-making for subsequent intelligent services, such as personalized learning source recommendation. Existing researchers have broadly studied KT and developed many effective methods. However, most of them assume that students' historical interactions are uniformly distributed in a continuous sequence, ignoring the fact that actual interaction sequences are organized based on a series of quizzes with clear boundaries, where interactions within a quiz are consecutively completed, but interactions across different quizzes are discrete and may be spaced over days. In this paper, we present the Quiz-based Knowledge Tracing (QKT) model to monitor students' knowledge states according to their quiz-based learning interactions. Specifically, as students' interactions within a quiz are continuous and have the same or similar knowledge concepts, we design the adjacent gate followed by a global average pooling layer to capture the intra-quiz short-term knowledge influence. Then, as various quizzes tend to focus on different knowledge concepts, we respectively measure the inter-quiz knowledge substitution by the gated recurrent unit and the inter-quiz knowledge complementarity by the self-attentive encoder with a novel recency-aware attention mechanism. Finally, we integrate the inter-quiz long-term knowledge substitution and complementarity across different quizzes to output students' evolving knowledge states. Extensive experimental results on three public real-world datasets demonstrate that QKT achieves state-of-the-art performance compared to existing methods. Further analyses confirm that QKT is promising in designing more effective quizzes., Comment: 12 pages

This paper proposes a method for generating images of customized objects specified by users. The method is based on a general framework that bypasses the lengthy optimization required by previous approaches, which often employ a per-object optimization paradigm. Our framework adopts an encoder to capture high-level identifiable semantics of objects, producing an object-specific embedding with only a single feed-forward pass. The acquired object embedding is then passed to a text-to-image synthesis model for subsequent generation. To effectively blend a object-aware embedding space into a well developed text-to-image model under the same generation context, we investigate different network designs and training strategies, and propose a simple yet effective regularized joint training scheme with an object identity preservation loss. Additionally, we propose a caption generation scheme that become a critical piece in fostering object specific embedding faithfully reflected into the generation process, while keeping control and editing abilities. Once trained, the network is able to produce diverse content and styles, conditioned on both texts and objects. We demonstrate through experiments that our proposed method is able to synthesize images with compelling output quality, appearance diversity, and object fidelity, without the need of test-time optimization. Systematic studies are also conducted to analyze our models, providing insights for future work.

Dissipaton theory had been proposed as an exact and nonperturbative approach to deal with open quantum system dynamics, where the influence of Gaussian environment is characterized by statistical quasi-particles named as dissipatons. In this work, we revisit the dissipaton equation of motion theory and establish an equivalent dissipatons-embedded quantum master equation (DQME), which gives rise to dissipatons as generalized Brownian particles. As explained in this work, the DQME supplies a direct approach to investigate the statistical characteristics of dissipatons and thus the physically supporting hybrid bath modes. Numerical demonstrations are carried out on the electron transfer model, exhibiting the transient statistical properties of the solvation coordinate., Comment: Correction of Eqs. (B11)-(B14)

In this work, we study a pseudo-Nambu-Goldstone boson (pNGB) dark matter model extended with a real scalar. The dark sector is assumed to be feebly coupled with the standard model (SM) via a Higgs portal, so that the pNGB dark matter is produced by the freeze-in mechanism. Since the production happened in a very high energy era, we introduce an extra scalar field which is weakly coupled to the SM for stablizing the electroweak vacuum. Our model can reproduce the correct relic abundance of dark matter favored by observations. In addition, we determine the evolution of couplings in higher energy scale by solving the renormalization group equations, and show that the self coupling of the real scalar, $\lambda_S$, and the mixing coupling between the Higgs field and the real scalar, $\lambda_{HS}$ are stringently constrained by the conditions of vacuum stability and couplings perturbativity up to the Planck scale. We also find that the relic abundance of DM is insensitive to the values of $\lambda_S$ and $\lambda_{HS}$ unless the dominant production processes are $\overline{H}+H(S+S)\to\phi+\phi$ via t- and u-channels., Comment: 30 pages, 10 figures

We present the quantum dynamics of a spin coupling to a bath of independent spins via the dissipaton equation of motion (DEOM) approach. The bath, characterized by a continuous spectral density function, is composed of spins that are independent level systems described by the su(2) Lie algebra. This represents an extreme class of anharmonic environment. Based on the conclusion drawn by Suarez and Silbey [J. Chem. Phys. 95, 9115 (1991)] and Makri [J. Chem. Phys. 111, 6164 (1999)] that the spin bath can be mapped to a Gaussian environment under its linear response limit, we derive the fluctuation-dissipation theorem (FDT) of the spin bath from a microscopic perspective, and generalize the discussion to the case of arbitrary bath spin quantum number S. Next, the time-domain Prony fitting decomposition scheme is applied to the bare-bath time correlation function (TCF) given by FDT to generate the exponential decay basis (or pseudo modes) for DEOM construction. The accuracy and efficiency of this strategy has been justified by a variety of numerical results. We envision this work provides new insights to extend the hierarchical equations of motion (HEOM) and DEOM approach to certain types of anharmonic enviroments with arbitrary TCF or spectral density

We review the derivation of the Boltzmann equation and its cosmological applications in this paper. The derivation of the Boltzmann equation, especially the collision term, is discussed in detail in the language of the quantum field theory without any assumption of the finite temperature system. We also discuss the integrated Boltzmann equation with the deal of the temperature parameter as an extension of the standard equation. Among a number of its cosmological applications, we mainly target two familiar examples, the dynamics of the dark matter abundance through the freeze-out/in process and a baryogenesis scenario. The formulations in those systems are briefly discussed with techniques in their calculations., Comment: 34 pages, 2 figures, 1 table

Task-oriented dialogue systems often assist users with personal or confidential matters. For this reason, the developers of such a system are generally prohibited from observing actual usage. So how can they know where the system is failing and needs more training data or new functionality? In this work, we study ways in which realistic user utterances can be generated synthetically, to help increase the linguistic and functional coverage of the system, without compromising the privacy of actual users. To this end, we propose a two-stage Differentially Private (DP) generation method which first generates latent semantic parses, and then generates utterances based on the parses. Our proposed approach improves MAUVE by 2.5$\times$ and parse tree function type overlap by 1.3$\times$ relative to current approaches for private synthetic data generation, improving both on fluency and semantic coverage. We further validate our approach on a realistic domain adaptation task of adding new functionality from private user data to a semantic parser, and show overall gains of 8.5% points in accuracy with the new feature., Comment: ACL 2023

A key missing capacity of current language models (LMs) is grounding to real-world environments. Most existing work for grounded language understanding uses LMs to directly generate plans that can be executed in the environment to achieve the desired effects. It thereby casts the burden of ensuring grammaticality, faithfulness, and controllability all on the LMs. We propose Pangu, a generic framework for grounded language understanding that capitalizes on the discriminative ability of LMs instead of their generative ability. Pangu consists of a symbolic agent and a neural LM working in a concerted fashion: The agent explores the environment to incrementally construct valid plans, and the LM evaluates the plausibility of the candidate plans to guide the search process. A case study on the challenging problem of knowledge base question answering (KBQA), which features a massive environment, demonstrates the remarkable effectiveness and flexibility of Pangu: A BERT-base LM is sufficient for setting a new record on standard KBQA datasets, and larger LMs further bring substantial gains. Pangu also enables, for the first time, effective few-shot in-context learning for KBQA with large LMs such as Codex., Comment: 18 pages, 6 figures, 6 tables; accepted to ACL'2023

Knowledge graph (KG) link prediction aims to infer new facts based on existing facts in the KG. Recent studies have shown that using the graph neighborhood of a node via graph neural networks (GNNs) provides more useful information compared to just using the query information. Conventional GNNs for KG link prediction follow the standard message-passing paradigm on the entire KG, which leads to superfluous computation, over-smoothing of node representations, and also limits their expressive power. On a large scale, it becomes computationally expensive to aggregate useful information from the entire KG for inference. To address the limitations of existing KG link prediction frameworks, we propose a novel retrieve-and-read framework, which first retrieves a relevant subgraph context for the query and then jointly reasons over the context and the query with a high-capacity reader. As part of our exemplar instantiation for the new framework, we propose a novel Transformer-based GNN as the reader, which incorporates graph-based attention structure and cross-attention between query and context for deep fusion. This simple yet effective design enables the model to focus on salient context information relevant to the query. Empirical results on two standard KG link prediction datasets demonstrate the competitive performance of the proposed method. Furthermore, our analysis yields valuable insights for designing improved retrievers within the framework., Comment: Accepted to CIKM'23; Published version DOI: https://doi.org/10.1145/3583780.3614769 ;12 pages, 4 figures

This study focuses on using large language models (LLMs) as a planner for embodied agents that can follow natural language instructions to complete complex tasks in a visually-perceived environment. The high data cost and poor sample efficiency of existing methods hinders the development of versatile agents that are capable of many tasks and can learn new tasks quickly. In this work, we propose a novel method, LLM-Planner, that harnesses the power of large language models to do few-shot planning for embodied agents. We further propose a simple but effective way to enhance LLMs with physical grounding to generate and update plans that are grounded in the current environment. Experiments on the ALFRED dataset show that our method can achieve very competitive few-shot performance: Despite using less than 0.5% of paired training data, LLM-Planner achieves competitive performance with recent baselines that are trained using the full training data. Existing methods can barely complete any task successfully under the same few-shot setting. Our work opens the door for developing versatile and sample-efficient embodied agents that can quickly learn many tasks. Website: https://dki-lab.github.io/LLM-Planner, Comment: 14 pages, 5 figures

Electronic correlation is a fundamental topic in many-electron systems. To characterize this correlation, one may introduce the concept of exchange-correlation hole. In this paper, we first briefly revisit its definition and relation to electron and geminal densities, followed by their intimate relations to copula functions in probability theory and statistics. We then propose a copula-based approach to estimate the exchange-correlation hole from the electron density. It is anticipated that the proposed scheme would become a promising ingredient towards the future development of strongly correlated electronic structure calculations., Comment: 5 pages

We present 230 GHz continuum polarization observations with the Atacama Large Milimeter/Submillimeter Array (ALMA) at a resolution of 0$\farcs1$ ($\sim 540$~au) in the high-mass star-forming regions W51 e2 and e8. These observations resolve a network of core-connecting dust lanes, marking a departure from earlier coarser more spherical continuum structures. At the same time, the cores do not appear to fragment further. Polarized dust emission is clearly detected. The inferred magnetic field orientations are prevailingly parallel to dust lanes. This key structural feature is analyzed together with the local gravitational vector field. The direction of local gravity is found to typically align with dust lanes. With these findings we derive a stability criterion that defines a maximum magnetic field strength that can be overcome by an observed magnetic field-gravity configuration. Equivalently, this defines a minimum field strength that can stabilize dust lanes against a radial collapse. We find that the detected dust lanes in W51 e2 and e8 are stable, hence possibly making them a fundamental component in the accretion onto central sources, providing support for massive star formation models without the need of large accretion disks. When comparing to coarser resolutions, covering the scales of envelope, global, and local collapse, we find recurring similarities in the magnetic field structures and their corresponding gravitational vector fields. These self-similar structures point at a multi-scale collapse-within-collapse scenario until finally the scale of core-accreting dust lanes is reached where gravity is entraining the magnetic field and aligning it with the dust lanes., Comment: accepted for publication in ApJ

Hand hygiene is a standard six-step hand-washing action proposed by the World Health Organization (WHO). However, there is no good way to supervise medical staff to do hand hygiene, which brings the potential risk of disease spread. Existing action assessment works usually make an overall quality prediction on an entire video. However, the internal structures of hand hygiene action are important in hand hygiene assessment. Therefore, we propose a novel fine-grained learning framework to perform step segmentation and key action scorer in a joint manner for accurate hand hygiene assessment. Existing temporal segmentation methods usually employ multi-stage convolutional network to improve the segmentation robustness, but easily lead to over-segmentation due to the lack of the long-range dependence. To address this issue, we design a multi-stage convolution-transformer network for step segmentation. Based on the observation that each hand-washing step involves several key actions which determine the hand-washing quality, we design a set of key action scorers to evaluate the quality of key actions in each step. In addition, there lacks a unified dataset in hand hygiene assessment. Therefore, under the supervision of medical staff, we contribute a video dataset that contains 300 video sequences with fine-grained annotations. Extensive experiments on the dataset suggest that our method well assesses hand hygiene videos and achieves outstanding performance.

Recent advances in deep learning have greatly propelled the research on semantic parsing. Improvement has since been made in many downstream tasks, including natural language interface to web APIs, text-to-SQL generation, among others. However, despite the close connection shared with these tasks, research on question answering over knowledge bases (KBQA) has comparatively been progressing slowly. We identify and attribute this to two unique challenges of KBQA, schema-level complexity and fact-level complexity. In this survey, we situate KBQA in the broader literature of semantic parsing and give a comprehensive account of how existing KBQA approaches attempt to address the unique challenges. Regardless of the unique challenges, we argue that we can still take much inspiration from the literature of semantic parsing, which has been overlooked by existing research on KBQA. Based on our discussion, we can better understand the bottleneck of current KBQA research and shed light on promising directions for KBQA to keep up with the literature of semantic parsing, particularly in the era of pre-trained language models., Comment: 19 pages, 3 figures; accepted to AKBC'22