Incentive engineering in microtask crowdsourcing

Crowdsourcing is emerging as an efficient approach to solve a wide variety of problems by engaging a large number of Internet users from many places in the world. However, the success of these systems relies critically on motivating the crowd to contribute, especially in microtask crowdsourcing contexts when the tasks are repetitive and easy for people to get bored. Given this, finding ways to efficiently incentivise participants in crowdsourcing projects in general and microtask crowdsourcing projects in particular is a major open challenge. Also, although there are numerous ways to incentivise participants in microtask crowdsourcing projects, the effectiveness of the incentives is likely to be different in different projects based on specific characteristics of those projects. Therefore, in a particular crowdsourcing project, a practical way to address the incentive problem is to choose a certain number of candidate incentives, then have a good strategy to select the most effective incentive at run time so as to maximise the cumulitive utility of the requesters within a given budget and time limit. We refer to this as the incentive selection problem (ISP). We present algorithms (HAIS and BOIS) to deal with the ISP by considering all characteristics of the problem. Specically, the algorithms make use of limited financial and time budgets to have a good exploration-exploitation balance. Also, they consider the group-based nature of the incentives (i.e., sampling two incentives with different group size yields two different number of samples) so as to make a good decision on how many times each incentive will be sampled at each time. By conducting extensive simulations, we show that our algorithms outperform state-of-the-art approaches in most cases. Also from the results of the simulations, practical usage of the two algorithms is discussed.