Stanford InfoLab Publication Server

Max Algorithms in Crowdsourcing Environments

Venetis, Petros and Garcia-Molina, Hector and Huang, Kerui and Polyzotis, Neoklis (2012) Max Algorithms in Crowdsourcing Environments. In: WWW, April 16-20, 2012, Lyon, France.




Our work investigates the problem of retrieving the maximum item from a set in crowdsourcing environments. We first develop parameterized families of max algorithms, that take as input a set of items and output an item from the set that is believed to be the maximum. Such max algorithms could, for instance, select the best Facebook profile that matches a given person or the best photo that describes a given restaurant. Then, we propose strategies that select appropriate max algorithm parameters. Our framework supports various human error and cost models and we consider many of them for our experiments. We evaluate under many metrics, both analytically and via simulations, the tradeoff between three quantities: (1) quality, (2) monetary cost, and (3) execution time. Also, we provide insights on the effectiveness of the strategies in selecting appropriate max algorithm parameters and guidelines for choosing max algorithms and strategies for each application.

Item Type:Conference or Workshop Item (Paper)
Uncontrolled Keywords:human computation, crowdsourcing, max algorithms, worker models, vote aggregation, plurality voting
Related URLs:Author Homepage
ID Code:1028
Deposited By:Petros Venetis
Deposited On:18 Feb 2012 18:19
Last Modified:22 Feb 2012 12:41

Download statistics

Repository Staff Only: item control page