Back to Search Start Over

An Empirical Comparison of Bias Reduction Methods on Real-World Problems in High-Stakes Policy Settings

Authors :
Kit T. Rodolfa
Hemank Lamba
Rayid Ghani
Source :
ACM SIGKDD Explorations Newsletter. 23:69-85
Publication Year :
2021
Publisher :
Association for Computing Machinery (ACM), 2021.

Abstract

Applications of machine learning (ML) to high-stakes policy settings -- such as education, criminal justice, healthcare, and social service delivery -- have grown rapidly in recent years, sparking important conversations about how to ensure fair outcomes from these systems. The machine learning research community has responded to this challenge with a wide array of proposed fairness-enhancing strategies for ML models, but despite the large number of methods that have been developed, little empirical work exists evaluating these methods in real-world settings. Here, we seek to fill this research gap by investigating the performance of several methods that operate at different points in the ML pipeline across four real-world public policy and social good problems. Across these problems, we find a wide degree of variability and inconsistency in the ability of many of these methods to improve model fairness, but post-processing by choosing group-specific score thresholds consistently removes disparities, with important implications for both the ML research community and practitioners deploying machine learning to inform consequential policy decisions.<br />17 pages, 9 figures, 2 tables

Details

ISSN :
19310153 and 19310145
Volume :
23
Database :
OpenAIRE
Journal :
ACM SIGKDD Explorations Newsletter
Accession number :
edsair.doi.dedup.....945b13b20a96b8c8051362e02472d81e
Full Text :
https://doi.org/10.1145/3468507.3468518