Back to Search
Start Over
Multi-Class Ground Truth Inference in Crowdsourcing with Clustering.
- Source :
-
IEEE Transactions on Knowledge & Data Engineering . Apr2016, Vol. 28 Issue 4, p1080-1085. 6p. - Publication Year :
- 2016
-
Abstract
- Due to low quality of crowdsourced labelers, the integrated label of each example is usually inferred from its multiple noisy labels provided by different labelers. This paper proposes a novel algorithm, Ground Truth Inference using Clustering (GTIC), to improve the quality of integrated labels for multi-class labeling. For a K labeling case, GTIC utilizes the multiple noisy label sets of examples to generate features. Then, it uses a K-Means algorithm to cluster all examples into K different groups, each of which is mapped to a specific class. Examples in the same cluster are assigned a corresponding class label. We compare GTIC with four existing multi-class ground truth inference algorithms, majority voting (MV), Dawid & Skene's (DS), ZenCrowd (ZC) and Spectral DS (SDS), on one synthetic and eight real-world datasets. Experimental results show that the performance of GTIC is significantly superior to the others in terms of both accuracy and M-AUC. Besides, the running time of GTIC is about twenty times faster than EM-based complicated inference algorithms. [ABSTRACT FROM PUBLISHER]
Details
- Language :
- English
- ISSN :
- 10414347
- Volume :
- 28
- Issue :
- 4
- Database :
- Academic Search Index
- Journal :
- IEEE Transactions on Knowledge & Data Engineering
- Publication Type :
- Academic Journal
- Accession number :
- 113814464
- Full Text :
- https://doi.org/10.1109/TKDE.2015.2504974