1. COUGH: A Challenge Dataset and Models for COVID-19 FAQ Retrieval
- Author
-
Zhang, Xinliang Frederick, Sun, Heming, Yue, Xiang, Lin, Simon, and Sun, Huan
- Subjects
FOS: Computer and information sciences ,Computer Science - Computation and Language ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,02 engineering and technology ,010501 environmental sciences ,01 natural sciences ,Computation and Language (cs.CL) ,Information Retrieval (cs.IR) ,Computer Science - Information Retrieval ,3. Good health ,0105 earth and related environmental sciences - Abstract
We present a large, challenging dataset, COUGH, for COVID-19 FAQ retrieval. Similar to a standard FAQ dataset, COUGH consists of three parts: FAQ Bank, Query Bank and Relevance Set. The FAQ Bank contains ~16K FAQ items scraped from 55 credible websites (e.g., CDC and WHO). For evaluation, we introduce Query Bank and Relevance Set, where the former contains 1,236 human-paraphrased queries while the latter contains ~32 human-annotated FAQ items for each query. We analyze COUGH by testing different FAQ retrieval models built on top of BM25 and BERT, among which the best model achieves 48.8 under P@5, indicating a great challenge presented by COUGH and encouraging future research for further improvement. Our COUGH dataset is available at https://github.com/sunlab-osu/covid-faq., Comment: EMNLP'21 Main Conference
- Published
- 2020
- Full Text
- View/download PDF