Search

This paper introduces the Eighth Dialog System Technology Challenge. In line with recent challenges, the eighth edition focuses on applying end-to-end dialog technologies in a pragmatic way for multi-domain task-completion, noetic response selection, audio visual scene-aware dialog, and schema-guided dialog state tracking tasks. This paper describes the task definition, provided datasets, baselines and evaluation set-up for each track. We also summarize the results of the submitted systems to highlight the overall trends of the state-of-the-art technologies for the tasks.

The 11 papers in this special section that were part of the Dialog System Technology Challenge. Research competitions have been a long-standing and valuable tradition in the speech and language community. They accelerate the development of new technologies by establishing a shared testbed, cross-validating many potential approaches, and attracting new researchers to the field. DSTC, the Dialog System Technology Challenge, has been a premier research competition for Dialog Systems since its inception in 2013. The earlier Dialog State Tracking Challenges focused on developing a single component for dialog state tracking on goaloriented human-machine conversations. Then, DSTC4 (2015) and DSTC5 (2016) introduced human-human conversations and started to offer multiple tasks not only for dialog state tracking, but also for other components in dialog systems as the pilot tasks. From the sixth challenge (2017), the DSTC has rebranded itself as “Dialog System Technology Challenge” and organized multiple main tracks in parallel to address a wider variety of end-to-end dialog related problems. In line with recent editions of this challenge, DSTC8 focuses on applying end-to-end technologies to Dialog Systems in a pragmatic way. We opened the call for track proposals to the dialog research community and finally held the four parallel main tracks: (1) Multi-domain Task Completion, (2) NOESIS II: Predicting Responses, Identifying Success, and Managing Complexity in Task-Oriented Dialogue, (3) Audio Visual SceneAware Dialog Track, and (4) Scalable Schema-Guided Dialogue State Tracking. This special issue presents more in-depth details of the challenge outcomes.