Back to Search
Start Over
An Empirical Comparison of Bias Reduction Methods on Real-World Problems in High-Stakes Policy Settings
- Source :
- ACM SIGKDD Explorations Newsletter. 23:69-85
- Publication Year :
- 2021
- Publisher :
- Association for Computing Machinery (ACM), 2021.
-
Abstract
- Applications of machine learning (ML) to high-stakes policy settings -- such as education, criminal justice, healthcare, and social service delivery -- have grown rapidly in recent years, sparking important conversations about how to ensure fair outcomes from these systems. The machine learning research community has responded to this challenge with a wide array of proposed fairness-enhancing strategies for ML models, but despite the large number of methods that have been developed, little empirical work exists evaluating these methods in real-world settings. Here, we seek to fill this research gap by investigating the performance of several methods that operate at different points in the ML pipeline across four real-world public policy and social good problems. Across these problems, we find a wide degree of variability and inconsistency in the ability of many of these methods to improve model fairness, but post-processing by choosing group-specific score thresholds consistently removes disparities, with important implications for both the ML research community and practitioners deploying machine learning to inform consequential policy decisions.<br />17 pages, 9 figures, 2 tables
- Subjects :
- FOS: Computer and information sciences
Computer Science - Machine Learning
Empirical comparison
Social work
Computer science
business.industry
Geography, Planning and Development
Public policy
02 engineering and technology
Data science
Pipeline (software)
Bias reduction
Machine Learning (cs.LG)
Computer Science - Computers and Society
Policy decision
020204 information systems
Computers and Society (cs.CY)
Health care
0202 electrical engineering, electronic engineering, information engineering
General Earth and Planetary Sciences
020201 artificial intelligence & image processing
business
Water Science and Technology
Criminal justice
Subjects
Details
- ISSN :
- 19310153 and 19310145
- Volume :
- 23
- Database :
- OpenAIRE
- Journal :
- ACM SIGKDD Explorations Newsletter
- Accession number :
- edsair.doi.dedup.....945b13b20a96b8c8051362e02472d81e
- Full Text :
- https://doi.org/10.1145/3468507.3468518