Search

New Intent Discovery (NID) aims to recognize known and infer new intent categories with the help of limited labeled and large-scale unlabeled data. The task is addressed as a feature-clustering problem and recent studies augment instance representation. However, existing methods fail to capture cluster-friendly representations, since they show less capability to effectively control and coordinate within-cluster and between-cluster distances. Tailored to the NID problem, we propose a Robust and Adaptive Prototypical learning (RAP) framework for globally distinct decision boundaries for both known and new intent categories. Specifically, a robust prototypical attracting learning (RPAL) method is designed to compel instances to gravitate toward their corresponding prototype, achieving greater within-cluster compactness. To attain larger between-cluster separation, another adaptive prototypical dispersing learning (APDL) method is devised to maximize the between-cluster distance from the prototype-to-prototype perspective. Experimental results evaluated on three challenging benchmarks (CLINC, BANKING, and StackOverflow) of our method with better cluster-friendly representation demonstrate that RAP brings in substantial improvements over the current state-of-the-art methods (even large language model) by a large margin (average +5.5% improvement)., Comment: COLING 2024

Recent LLM-based Text-to-SQL methods usually suffer from significant performance degradation on "huge" databases and complex user questions that require multi-step reasoning. Moreover, most existing methods neglect the crucial significance of LLMs utilizing external tools and model collaboration. To address these challenges, we introduce MAC-SQL, a novel LLM-based multi-agent collaborative framework. Our framework comprises a core decomposer agent for Text-to-SQL generation with few-shot chain-of-thought reasoning, accompanied by two auxiliary agents that utilize external tools or models to acquire smaller sub-databases and refine erroneous SQL queries. The decomposer agent collaborates with auxiliary agents, which are activated as needed and can be expanded to accommodate new features or tools for effective Text-to-SQL parsing. In our framework, We initially leverage GPT-4 as the strong backbone LLM for all agent tasks to determine the upper bound of our framework. We then fine-tune an open-sourced instruction-followed model, SQL-Llama, by leveraging Code Llama 7B, to accomplish all tasks as GPT-4 does. Experiments show that SQL-Llama achieves a comparable execution accuracy of 43.94, compared to the baseline accuracy of 46.35 for vanilla GPT-4. At the time of writing, MAC-SQL+GPT-4 achieves an execution accuracy of 59.59 when evaluated on the BIRD benchmark, establishing a new state-of-the-art (SOTA) on its holdout test set (https://github.com/wbbeyourself/MAC-SQL).

A customer service platform system with a core text semantic similarity (STS) task faces two urgent challenges: Firstly, one platform system needs to adapt to different domains of customers, i.e., different domains adaptation (DDA). Secondly, it is difficult for the model of the platform system to distinguish sentence pairs that are literally close but semantically different, i.e., hard negative samples. In this paper, we propose an incorporation external keywords matrices model (IEKM) to address these challenges. The model uses external tools or dictionaries to construct external matrices and fuses them to the self-attention layers of the Transformer structure through gating units, thus enabling flexible corrections to the model results. We evaluate the method on multiple datasets and the results show that our method has improved performance on all datasets. To demonstrate that our method can effectively solve all the above challenges, we conduct a flexible correction experiment, which results in an increase in the F1 value from 56.61 to 73.53. Our code will be publicly available.

Existing knowledge-grounded conversation systems generate responses typically in a retrieve-then-generate manner. They require a large knowledge base and a strong knowledge retrieval component, which is time- and resource-consuming. In this paper, we address the challenge by leveraging the inherent knowledge encoded in the pre-trained language models (PLMs). We propose Knowledgeable Prefix Tuning (KnowPrefix-Tuning), a two-stage tuning framework, bypassing the retrieval process in a knowledge-grounded conversation system by injecting prior knowledge into the lightweight knowledge prefix. The knowledge prefix is a sequence of continuous knowledge-specific vectors that can be learned during training. In addition, we propose a novel interactive re-parameterization mechanism that allows the prefix to interact fully with the PLM during the optimization of response generation. Experimental results demonstrate that KnowPrefix-Tuning outperforms fine-tuning and other lightweight tuning approaches, and performs comparably with strong retrieval-based baselines while being $3\times$ faster during inference., Comment: Accepted by ECML-PKDD 2023 (Research Track)

Retrieval-enhanced text generation has shown remarkable progress on knowledge-intensive language tasks, such as open-domain question answering and knowledge-enhanced dialogue generation, by leveraging passages retrieved from a large passage corpus for delivering a proper answer given the input query. However, the retrieved passages are not ideal for guiding answer generation because of the discrepancy between retrieval and generation, i.e., the candidate passages are all treated equally during the retrieval procedure without considering their potential to generate a proper answer. This discrepancy makes a passage retriever deliver a sub-optimal collection of candidate passages to generate the answer. In this paper, we propose the GeneRative Knowledge Improved Passage Ranking (GripRank) approach, addressing the above challenge by distilling knowledge from a generative passage estimator (GPE) to a passage ranker, where the GPE is a generative language model used to measure how likely the candidate passages can generate the proper answer. We realize the distillation procedure by teaching the passage ranker learning to rank the passages ordered by the GPE. Furthermore, we improve the distillation quality by devising a curriculum knowledge distillation mechanism, which allows the knowledge provided by the GPE can be progressively distilled to the ranker through an easy-to-hard curriculum, enabling the passage ranker to correctly recognize the provenance of the answer from many plausible candidates. We conduct extensive experiments on four datasets across three knowledge-intensive language tasks. Experimental results show advantages over the state-of-the-art methods for both passage ranking and answer generation on the KILT benchmark., Comment: CIKM 2023

Context-dependent Text-to-SQL aims to translate multi-turn natural language questions into SQL queries. Despite various methods have exploited context-dependence information implicitly for contextual SQL parsing, there are few attempts to explicitly address the dependencies between current question and question context. This paper presents QURG, a novel Question Rewriting Guided approach to help the models achieve adequate contextual understanding. Specifically, we first train a question rewriting model to complete the current question based on question context, and convert them into a rewriting edit matrix. We further design a two-stream matrix encoder to jointly model the rewriting relations between question and context, and the schema linking relations between natural language and structured schema. Experimental results show that QURG significantly improves the performances on two large-scale context-dependent datasets SParC and CoSQL, especially for hard and long-turn questions.

Two-dimensional material-based field effect transistors (2DM-FETs) are playing a revolutionary role in electronic devices. However, after years of development, no device model can match the Pao-Sah model for standard silicon-based transistors in terms of physical accuracy and computational efficiency to support large-scale integrated circuit design. One remaining critical obstacle is the contacts of 2DM-FETs. In order to self-consistently include the contact effect in the current model, it is necessary to perform self-consistent calculations, which is a fatal flaw for applications that prioritize efficiency. Here, we report that the Landauer-QFLPS model effectively overcomes the above contradiction, where QFLPS means quasi-Fermi-level phase space theory. By connecting the physical pictures of the contact and the intrinsic channel part, we have successfully derived a drain-source current formula including the contact effect. To verify the model, we prepared transistors based on two typical 2DMs, black phosphorus (BP) and molybdenum disulfide (MoS2), the former having ambipolar transport and the latter showing electron-dominant unipolar transport. The proposed new formula could describe both 2DM-FETs with Schottky or Ohmic contacts. Moreover, compared with traditional methods, the proposed model has the advantages of accuracy and efficiency, especially in describing non-monotonic drain conductance characteristics, because the contact effect is self-consistently and compactly packaged as an exponential term. More importantly, we also examined the model at the circuit level. Here, we fabricated a three-bit threshold inverter quantizer circuit based on ambipolar-BP process and experimentally demonstrated that the model can accurately predict the circuit performance. This industry-benign 2DM-FET model is supposed to be very useful for the development of 2DM-FET-based integrated circuits.

Two-dimensional materials-based field-effect transistors (2DM-FETs) exhibit both ambipolar and unipolar transport types. To physically and compactly cover both cases, we put forward a quasi-Fermi-level phase space (QFLPS) approach to model the ambipolar effect in our previous work. This work aims to further improve the QFLPS model's numerical aspect so that the model can be implanted into the standard circuit simulator. We first rigorously derive the integral-free formula for the drain-source current to achieve this goal. It is more friendly to computation than the integral form. Besides, it explicitly gives the correlation terms between the electron and hole components. Secondly, to work out the boundary values required by the new expressions, we develop a fast evaluation algorithm for the surface electrostatic potential based on the zero-temperature limit property of the 2DM-FET system. By calibrating the model with the realistic device data of black phosphorus (BP) and monolayer molybdenum disulfide (ML-MoS2) FETs, the completed algorithm is tested against practical cases. The results show a typical superiority to the benchmark algorithm by two orders of magnitude in time consumption can be achieved while keeping a high accuracy with 7 to 9 significant digits., Comment: 17 pages, 9 figures

Context-dependent text-to-SQL is the task of translating multi-turn questions into database-related SQL queries. Existing methods typically focus on making full use of history context or previously predicted SQL for currently SQL parsing, while neglecting to explicitly comprehend the schema and conversational dependency, such as co-reference, ellipsis and user focus change. In this paper, we propose CQR-SQL, which uses auxiliary Conversational Question Reformulation (CQR) learning to explicitly exploit schema and decouple contextual dependency for SQL parsing. Specifically, we first present a schema enhanced recursive CQR method to produce domain-relevant self-contained questions. Secondly, we train CQR-SQL models to map the semantics of multi-turn questions and auxiliary self-contained questions into the same latent space through schema grounding consistency task and tree-structured SQL parsing consistency task, which enhances the abilities of SQL parsing by adequately contextual understanding. At the time of writing, our CQR-SQL achieves new state-of-the-art results on two context-dependent text-to-SQL benchmarks SParC and CoSQL., Comment: Accepted at EMNLP 2022 (findings)

We propose a multi-channel speech enhancement approach with a novel two-stage feature fusion method and a pre-trained acoustic model in a multi-task learning paradigm. In the first fusion stage, the time-domain and frequency-domain features are extracted separately. In the time domain, the multi-channel convolution sum (MCS) and the inter-channel convolution differences (ICDs) features are computed and then integrated with the first 2-D convolutional layer, while in the frequency domain, the log-power spectra (LPS) features from both original channels and super-directive beamforming outputs are combined with a second 2-D convolutional layer. To fully integrate the rich information of multi-channel speech, i.e. time-frequency domain features and the array geometry, we apply a third 2-D convolutional layer in the second fusion stage to obtain the final convolutional features. Furthermore, we propose to use a fixed clean acoustic model trained with the end-to-end lattice-free maximum mutual information criterion to enforce the enhanced output to have the same distribution as the clean waveform to alleviate the over-estimation problem of the enhancement task and constrain distortion. On the Task1 development dataset of ConferencingSpeech 2021 challenge, a PESQ improvement of 0.24 and 0.19 is attained compared to the official baseline and a recently proposed multi-channel separation method., Comment: 7 pages, 3 figures, accepted to APSIPA 2021, revised

In multi-label text classification (MLTC), each given document is associated with a set of correlated labels. To capture label correlations, previous classifier-chain and sequence-to-sequence models transform MLTC to a sequence prediction task. However, they tend to suffer from label order dependency, label combination over-fitting and error propagation problems. To address these problems, we introduce a novel approach with multi-task learning to enhance label correlation feedback. We first utilize a joint embedding (JE) mechanism to obtain the text and label representation simultaneously. In MLTC task, a document-label cross attention (CA) mechanism is adopted to generate a more discriminative document representation. Furthermore, we propose two auxiliary label co-occurrence prediction tasks to enhance label correlation learning: 1) Pairwise Label Co-occurrence Prediction (PLCP), and 2) Conditional Label Co-occurrence Prediction (CLCP). Experimental results on AAPD and RCV1-V2 datasets show that our method outperforms competitive baselines by a large margin. We analyze low-frequency label performance, label dependency, label combination diversity and coverage speed to show the effectiveness of our proposed method on label correlation learning., Comment: Accepted by ACL 2021 (Finding)

We present the $W_{1+\infty}$ constraints for the Gaussian Hermitian matrix model, where the constructed constraint operators yield the $W_{1+\infty}$ $n$-algebra. For the Virasoro constraints, we note that the constraint operators give the null 3-algebra. With the help of our Virasoro constraints, we derive a new effective formula for correlators in the Gaussian Hermitian matrix model., Comment: 13 pages, accepted for publication in Physics Letters B

We construct the multi-variable realizations of the $W_{1+\infty}$ algebra such that they lead to the $W_{1+\infty}$ $n$-algebra. Based on our realizations of the $W_{1+\infty}$ algebra, we derive the $W_{1+\infty}$ constraints for the hermitian one-matrix model. The constraint operators yield not only the $W_{1+\infty}$ algebra but also the closed $W_{1+\infty}$ $n$-algebra., Comment: 14 pages

Machine reading comprehension (MRC) requires reasoning about both the knowledge involved in a document and knowledge about the world. However, existing datasets are typically dominated by questions that can be well solved by context matching, which fail to test this capability. To encourage the progress on knowledge-based reasoning in MRC, we present knowledge-based MRC in this paper, and build a new dataset consisting of 40,047 question-answer pairs. The annotation of this dataset is designed so that successfully answering the questions requires understanding and the knowledge involved in a document. We implement a framework consisting of both a question answering model and a question generation model, both of which take the knowledge extracted from the document as well as relevant facts from an external knowledge base such as Freebase/ProBase/Reverb/NELL. Results show that incorporating side information from external KB improves the accuracy of the baseline question answer system. We compare it with a standard MRC model BiDAF, and also provide the difficulty of the dataset and lay out remaining challenges.

In this paper, we study automatic keyphrase generation. Although conventional approaches to this task show promising results, they neglect correlation among keyphrases, resulting in duplication and coverage issues. To solve these problems, we propose a new sequence-to-sequence architecture for keyphrase generation named CorrRNN, which captures correlation among multiple keyphrases in two ways. First, we employ a coverage vector to indicate whether the word in the source document has been summarized by previous phrases to improve the coverage for keyphrases. Second, preceding phrases are taken into account to eliminate duplicate phrases and improve result coherence. Experiment results show that our model significantly outperforms the state-of-the-art method on benchmark datasets in terms of both accuracy and diversity., Comment: EMNLP 2018

In this paper, we present a generative model to generate a natural language sentence describing a table region, e.g., a row. The model maps a row from a table to a continuous vector and then generates a natural language sentence by leveraging the semantics of a table. To deal with rare words appearing in a table, we develop a flexible copying mechanism that selectively replicates contents from the table in the output sequence. Extensive experiments demonstrate the accuracy of the model and the power of the copying mechanism. On two synthetic datasets, WIKIBIO and SIMPLEQUESTIONS, our model improves the current state-of-the-art BLEU-4 score from 34.70 to 40.26 and from 33.32 to 39.12, respectively. Furthermore, we introduce an open-domain dataset WIKITABLETEXT including 13,318 explanatory sentences for 4,962 tables. Our model achieves a BLEU-4 score of 38.23, which outperforms template based and language model based approaches., Comment: 9 pages, 4 figures. This paper has been published by AAAI2018

We present assertion based question answering (ABQA), an open domain question answering task that takes a question and a passage as inputs, and outputs a semi-structured assertion consisting of a subject, a predicate and a list of arguments. An assertion conveys more evidences than a short answer span in reading comprehension, and it is more concise than a tedious passage in passage-based QA. These advantages make ABQA more suitable for human-computer interaction scenarios such as voice-controlled speakers. Further progress towards improving ABQA requires richer supervised dataset and powerful models of text understanding. To remedy this, we introduce a new dataset called WebAssertions, which includes hand-annotated QA labels for 358,427 assertions in 55,960 web passages. To address ABQA, we develop both generative and extractive approaches. The backbone of our generative approach is sequence to sequence learning. In order to capture the structure of the output assertion, we introduce a hierarchical decoder that first generates the structure of the assertion and then generates the words of each field. The extractive approach is based on learning to rank. Features at different levels of granularity are designed to measure the semantic relevance between a question and an assertion. Experimental results show that our approaches have the ability to infer question-aware assertions from a passage. We further evaluate our approaches by incorporating the ABQA results as additional features in passage-based QA. Results on two datasets show that ABQA features significantly improve the accuracy on passage-based~QA., Comment: To be published at AAAI 2018

Understanding the connections between unstructured text and semi-structured table is an important yet neglected problem in natural language processing. In this work, we focus on content-based table retrieval. Given a query, the task is to find the most relevant table from a collection of tables. Further progress towards improving this area requires powerful models of semantic matching and richer training and evaluation resources. To remedy this, we present a ranking based approach, and implement both carefully designed features and neural network architectures to measure the relevance between a query and the content of a table. Furthermore, we release an open-domain dataset that includes 21,113 web queries for 273,816 tables. We conduct comprehensive experiments on both real world and synthetic datasets. Results verify the effectiveness of our approach and present the challenges for this task.

We study the problem of joint question answering (QA) and question generation (QG) in this paper. Our intuition is that QA and QG have intrinsic connections and these two tasks could improve each other. On one side, the QA model judges whether the generated question of a QG model is relevant to the answer. On the other side, the QG model provides the probability of generating a question given the answer, which is a useful evidence that in turn facilitates QA. In this paper we regard QA and QG as dual tasks. We propose a training framework that trains the models of QA and QG simultaneously, and explicitly leverages their probabilistic correlation to guide the training process of both models. We implement a QG model based on sequence-to-sequence learning, and a QA model based on recurrent neural network. As all the components of the QA and QG models are differentiable, all the parameters involved in these two models could be conventionally learned with back propagation. We conduct experiments on three datasets. Empirical results show that our training framework improves both QA and QG tasks. The improved QA model performs comparably with strong baseline approaches on all three datasets.

We present the nontrivial $W_{1+\infty}$ $n$-algebra and analyze its remarkable properties. We investigate the $W_{1+\infty}$ $n$-algebra in the Landau problem and discuss the realization of the classical $w_{\infty}$ 3-algebra. Furthermore, we discuss the case of the many-body system in the lowest Landau level and derive the constraints for correlation functions of the vertex operators., Comment: 15 pages

n-ary algebras have played important roles in mathematics and mathematical physics. The purpose of this paper is to construct a deformation of Virasoro-Witt n-algebra based on an oscillator realization with two independent parameters (p, q) and investigate its n-Lie subalgebra.

We investigate the quantum Calogero-Moser model and reveal its hidden symmetries, i.e., the $W_{1+\infty}$ and Virasoro-Witt 3-algebras. In the large $N$ limit, we note that these two infinite dimensional 3-algebras reduce to the $w_{\infty}$ and special Virasoro-Witt 3-algebras which satisfy the fundamental identity condition, respectively., Comment: This paper has been withdrawn by the author due to a error in algebra

The $q$-deformation of the infinite-dimensional $n$-algebra is investigated. Based on the structure of the $q$-deformed Virasoro-Witt algebra, we derive a nontrivial $q$-deformed Virasoro-Witt $n$-algebra which is nothing but a sh-$n$-Lie algebra. Furthermore in terms of the pseud-differential operators on the quantum plane, we construct the (co)sine $n$-algebra and the $q$-deformed $SDiff(T^2)$ $n$-algebra. We prove that they are the sh-$n$-Lie algebras for the case of even $n$. An explicit physical realization of the (co)sine $n$-algebra is given., Comment: 22 pages

IHEP, China is constructing a 100 MeV / 100 kW electron Linac for NSC KIPT, Ukraine. This linac will be used as the driver of a neutron source based on a subcritical assembly. In 2012, the injector part of the accelerator was pre-installed as a testing facility in the experimental hall #2 of IHEP. The injector beam and key hardware testing results were met the design goal. Recently, the injector testing facility was disassembled and all of the components for the whole accelerator have been shipped to Ukraine from China by ocean shipping. The installation of the whole machine in KIPT will be started in June, 2013. The construction progress, the design and testing results of the injector beam and key hardware are presented., Comment: 8 pages, 25 figures, submitted and accepted by Chinese Physics C (Formerly High Energy Physics and Nuclear Physics)