Search

Task-oriented dialogues often require agents to enact complex, multi-step procedures in order to meet user requests. While large language models have found success automating these dialogues in constrained environments, their widespread deployment is limited by the substantial quantities of task-specific data required for training. The following paper presents a data-efficient solution to constructing dialogue systems, leveraging explicit instructions derived from agent guidelines, such as company policies or customer service manuals. Our proposed Knowledge-Augmented Dialogue System (KADS) combines a large language model with a knowledge retrieval module that pulls documents outlining relevant procedures from a predefined set of policies, given a user-agent interaction. To train this system, we introduce a semi-supervised pre-training scheme that employs dialogue-document matching and action-oriented masked language modeling with partial parameter freezing. We evaluate the effectiveness of our approach on prominent task-oriented dialogue datasets, Action-Based Conversations Dataset and Schema-Guided Dialogue, for two dialogue tasks: action state tracking and workflow discovery. Our results demonstrate that procedural knowledge augmentation improves accuracy predicting in- and out-of-distribution actions while preserving high performance in settings with low or sparse data.

Language use differs dramatically from context to context. To some degree, modern language models like GPT-3 are able to account for such variance by conditioning on a string of previous input text, or prompt. Yet prompting is ineffective when contexts are sparse, out-of-sample, or extra-textual; for instance, accounting for when and where the text was produced or who produced it. In this paper, we introduce the mixed-effects transformer (MET), a novel approach for learning hierarchically-structured prefixes -- lightweight modules prepended to the input -- to account for structured variation. Specifically, we show how the popular class of mixed-effects models may be extended to transformer-based architectures using a regularized prefix-tuning procedure with dropout. We evaluate this approach on several domain-adaptation benchmarks, finding that it efficiently adapts to novel contexts with minimal data while still effectively generalizing to unseen contexts.

An overarching goal of natural language processing is to enable machines to communicate seamlessly with humans. However, natural language can be ambiguous or unclear. In cases of uncertainty, humans engage in an interactive process known as repair: asking questions and seeking clarification until their uncertainty is resolved. We propose a framework for building a visually grounded question-asking model capable of producing polar (yes-no) clarification questions to resolve misunderstandings in dialogue. Our model uses an expected information gain objective to derive informative questions from an off-the-shelf image captioner without requiring any supervised question-answer data. We demonstrate our model's ability to pose questions that improve communicative success in a goal-oriented 20 questions game with synthetic and human answerers., Comment: EMNLP 2021

To be good conversational partners, natural language processing (NLP) systems should be trained to produce contextually useful utterances. Prior work has investigated training NLP systems with communication-based objectives, where a neural listener stands in as a communication partner. However, these systems commonly suffer from semantic drift where the learned language diverges radically from natural language. We propose a method that uses a population of neural listeners to regularize speaker training. We first show that language drift originates from the poor uncertainty calibration of a neural listener, which makes high-certainty predictions on novel sentences. We explore ensemble- and dropout-based populations of listeners and find that the former results in better uncertainty quantification. We evaluate both population-based objectives on reference games, and show that the ensemble method with better calibration enables the speaker to generate pragmatic utterances while scaling to a large vocabulary and generalizing to new games and listeners., Comment: Findings of EMNLP 2021 Code: https://github.com/rosewang2008/calibrate_your_listeners

A hallmark of human language is the ability to effectively and efficiently convey contextually relevant information. One theory for how humans reason about language is presented in the Rational Speech Acts (RSA) framework, which captures pragmatic phenomena via a process of recursive social reasoning (Goodman & Frank, 2016). However, RSA represents ideal reasoning in an unconstrained setting. We explore the idea that speakers might learn to amortize the cost of RSA computation over time by directly optimizing for successful communication with an internal listener model. In simulations with grounded neural speakers and listeners across two communication game datasets representing synthetic and human-generated data, we find that our amortized model is able to quickly generate language that is effective and concise across a range of contexts, without the need for explicit pragmatic reasoning., Comment: Accepted to CogSci 2020

People with disabilities are present and visible at universities and postsecondary institutions as never before. This reflects changes in societal attitudes, law, public policy, and government programs, and, perhaps most important, the views disabled people have of themselves. The growing presence of this new "minority" on campus poses challenges to all postsecondary institutions. This information package contains essays, reprints, and resources designed to assist postsecondary institutions to move beyond basic compliance and to include disabled persons in all aspects of campus life. It is designed to offer perspectives, strategies, and resources that individuals can use to advocate for the inclusion of people with disabilities at universities and postsecondary institutions. This package uses Section 504 and the Americans with Disabilities Act (ADA) as a starting point for discussions of the inclusion of persons with disabilities, followed by discussion of the case for moving beyond compliance and various ways postsecondary institutions, faculty, and students can accomplish this. Specific accommodations for the disabled are presented, including a program that involves young adults with intellectual and developmental disabilities in campus life. Reprints of published articles that address different ways in which the visibility of disability on campus can be raised are included. The report concludes with annotations of selected published sources and additional resources. (Contains 4 notes.) [This publication was written with Jagdish Chander, Eugene Marcus, Michael Schwartz, Valerie Smith, Cheryl Spear, and Rachael Zubal-Ruggieri.]

As part of a 2-year study to plan, develop, and offer relevant continuing education programs for women in Pensacola, a Delphi study was conducted to determine the educational needs and priorities of different types of women living in Pensacola. The Delphi method was selected because it is considered an effective way of eliciting the opinions of a variety of people free of group dynamics, because it has been shown to produce a superior amount and quality of information relative to the resources and time expended, and because it seemed an innovative way to develop a community of women who might in the future work together on women's programs in Pensacola. Four iterative questionnaires were developed and distributed over a 7-month period. A respondent panel of 56 women reviewed and ranked objectives and resources available (resources we have) and resources not available (resources we need) required to achieve the objectives. The respondents also recommended, reviewed, and voted on specific program suggestions that could be developed during the second year of the project. The study achieved its primary aim: To offer relevant program suggestions for the Continuing Education for Women Program in Pensacola. The report provides an overview of the Delphi technique, summary of the methods and questionnaires, and data summary. Appendix 1 lists available and needed resources and recommended programs under each of 10 objectives: Day care facilities available as needed, increased participation of women in politics, community awareness programs for women, a women's center, creation of part-time job opportunities for women, programs to increase women's awareness of Federal and State legislation which protect their interests and rights, compilation of a community and regional directory of existing opportunities and resources, career awareness throughout the elementary and secondary schools, enforcement of equitable credit regulations, and public school education in parenting. Second-year program plans are outlined in child care, health, politics, and employment. (JT)