Stanford InfoLab Publication Server

Max Algorithms in Crowdsourcing Environments

Venetis, Petros and Garcia-Molina, Hector and Huang, Kerui and Polyzotis, Neoklis (2011) Max Algorithms in Crowdsourcing Environments. Technical Report. Stanford InfoLab.




Our work investigates the problem of retrieving the maximum item from a set in crowdsourcing environments. We first develop parameterized families of max algorithms, that take as input a set of items and output an item from the set that is believed to be the maximum. Such max algorithms could, for instance, select the best Facebook profile that matches a given person or the best photo that describes a given restaurant. Then, we propose strategies that select appropriate max algorithm parameters. Our framework supports various human error and cost models and we consider many of them for our experiments. We evaluate under many metrics, both analytically and via simulations, the tradeoff between three quantities: (1) quality, (2) monetary cost, and (3) execution time. Also, we provide insights on the effectiveness of the strategies in selecting appropriate max algorithm parameters and guidelines for choosing max algorithms and strategies for each application.

Item Type:Techreport (Technical Report)
Uncontrolled Keywords:human computation, crowdsourcing, max algorithms, worker models, vote aggregation, plurality voting
Related URLs:Author Homepage
ID Code:1017
Deposited By:Petros Venetis
Deposited On:07 Nov 2011 10:51
Last Modified:18 Feb 2012 18:14

Download statistics

Repository Staff Only: item control page