Back to Search
Start Over
Efficient crowdsourcing of crowd-generated microtasks
- Source :
- PLoS ONE, PLoS ONE, Vol 15, Iss 12, p e0244245 (2020)
- Publication Year :
- 2019
-
Abstract
- Allowing members of the crowd to propose novel microtasks for one another is an effective way to combine the efficiencies of traditional microtask work with the inventiveness and hypothesis generation potential of human workers. However, microtask proposal leads to a growing set of tasks that may overwhelm limited crowdsourcer resources. Crowdsourcers can employ methods to utilize their resources efficiently, but algorithmic approaches to efficient crowdsourcing generally require a fixed task set of known size. In this paper, we introduce *cost forecasting* as a means for a crowdsourcer to use efficient crowdsourcing algorithms with a growing set of microtasks. Cost forecasting allows the crowdsourcer to decide between eliciting new tasks from the crowd or receiving responses to existing tasks based on whether or not new tasks will cost less to complete than existing tasks, efficiently balancing resources as crowdsourcing occurs. Experiments with real and synthetic crowdsourcing data show that cost forecasting leads to improved accuracy. Accuracy and efficiency gains for crowd-generated microtasks hold the promise to further leverage the creativity and wisdom of the crowd, with applications such as generating more informative and diverse training data for machine learning applications and improving the performance of user-generated content and question-answering platforms.<br />12 pages, 5 figures
- Subjects :
- FOS: Computer and information sciences
Computer Science - Machine Learning
Computer science
Computer Science - Human-Computer Interaction
Social Sciences
02 engineering and technology
Markov Processes
computer.software_genre
Machine Learning (cs.LG)
Machine Learning
Creativity
Mathematical and Statistical Techniques
Sociology
Statistics - Machine Learning
Task Performance and Analysis
0202 electrical engineering, electronic engineering, information engineering
Psychology
Problem Solving
Multidisciplinary
Training set
Applied Mathematics
Simulation and Modeling
Statistics
Social system
Physical Sciences
symbols
Social Systems
Medicine
Crowdsourcing
020201 artificial intelligence & image processing
Algorithms
Research Article
Computer and Information Sciences
Science
Markov process
Machine Learning (stat.ML)
Machine learning
Research and Analysis Methods
Statistics - Applications
Human-Computer Interaction (cs.HC)
symbols.namesake
Machine Learning Algorithms
Artificial Intelligence
020204 information systems
Wisdom of the crowd
Leverage (statistics)
Humans
Applications (stat.AP)
Computer Simulation
Statistical Methods
Set (psychology)
business.industry
Cognitive Psychology
Biology and Life Sciences
Probability Theory
Probability Distribution
Task (computing)
Cognitive Science
Artificial intelligence
business
computer
Mathematics
Forecasting
Neuroscience
Subjects
Details
- Language :
- English
- Database :
- OpenAIRE
- Journal :
- PLoS ONE, PLoS ONE, Vol 15, Iss 12, p e0244245 (2020)
- Accession number :
- edsair.doi.dedup.....ec7d9f4c19daf06a838568a7ccf2bde7