The methodology of studying fairness perceptions in Artificial Intelligence: Contrasting CHI and FAccT.

The topic of algorithmic fairness is of increasing importance to the Human–Computer Interaction research community following accumulating concerns regarding the use and deployment of Artificial Intelligence-based systems. How we conduct research on algorithmic fairness directly influences our inferences and conclusions regarding algorithmic fairness. To better understand the methodological decisions of studies focused on people's perceptions of algorithmic fairness, we systematic analysed relevant papers from the CHI and FAccT conferences. We identified 200 relevant papers published between 1993 and 2022 and assessed their study design, participant sample, and geographical location of participants and authors. Our results highlight that studies are predominantly cross-sectional, cover a wide range of participant roles, and that both authors and participants are primarily from the United States. Based on these findings, we reflect on the potential pitfalls and shortcomings in how the community studies algorithmic fairness. • Mapping of CHI and FAccT papers on algorithmic fairness published 1993–2022. • Over one-third of papers miss details on participant locality and compensation. • Algorithmic fairness research largely restricted to US and other Western countries. • Limited cross-country collaboration, most study samples from a single country. • Remote studies are most common, longitudinal studies are relatively rare. [ABSTRACT FROM AUTHOR]