Mitigate Gender Bias using Negative Multi-Task Learning

Deep learning models have shown their great performances in natural language processing tasks. While much attention has been paid towards improvements in utility, privacy leakage and social bias are two major concerns arising in trained models. In this paper, we protect individual privacy and mitigate gender bias on classification models simultaneously. First, we propose a selective privacy-preserving method that only obscure individuals' sensitive information by adding noise on word embeddings. Then we propose a negative multi-task learning framework to mitigate the gender bias which contains a main task and a gender prediction task. The main task uses a positive loss constraint to ensure utility while the gender prediction task applies a negative loss constraint to remove gender-specific features. We analyze two existing word embeddings and evaluate them on sentiment analysis and medical text classification tasks. Our experimental results show that our negative multi-task learning framework can mitigate the gender bias while keeping models’ utility on both sentiment analysis and medical text classification.