1. Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP
- Author
-
Han, Xudong, Baldwin, Timothy, and Cohn, Trevor
- Subjects
FOS: Computer and information sciences ,Computer Science - Computers and Society ,Computer Science - Machine Learning ,Computer Science - Computation and Language ,Computers and Society (cs.CY) ,Computation and Language (cs.CL) ,Machine Learning (cs.LG) - Abstract
Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. However current progress is hampered by a plurality of definitions of bias, means of quantification, and oftentimes vague relation between debiasing algorithms and theoretical measures of bias. This paper seeks to clarify the current situation and plot a course for meaningful progress in fair learning, with two key contributions: (1) making clear inter-relations among the current gamut of methods, and their relation to fairness theory; and (2) addressing the practical problem of model selection, which involves a trade-off between fairness and accuracy and has led to systemic issues in fairness research. Putting them together, we make several recommendations to help shape future work., EACL 2023
- Published
- 2023