Venetis, Petros and Garcia-Molina, Hector and Huang, Kerui and Polyzotis, Neoklis (2011) Max Algorithms in Crowdsourcing Environments. Technical Report. Stanford InfoLab.
Our work investigates the problem of retrieving the maximum item from a set in crowdsourcing environments. We first develop parameterized families of max algorithms, that take as input a set of items and output an item from the set that is believed to be the maximum. Such max algorithms could, for instance, select the best Facebook profile that matches a given person or the best photo that describes a given restaurant. Then, we propose strategies that select appropriate max algorithm parameters. Our framework supports various human error and cost models and we consider many of them for our experiments. We evaluate under many metrics, both analytically and via simulations, the tradeoff between three quantities: (1) quality, (2) monetary cost, and (3) execution time. Also, we provide insights on the effectiveness of the strategies in selecting appropriate max algorithm parameters and guidelines for choosing max algorithms and strategies for each application.
|Item Type:||Techreport (Technical Report)|
|Uncontrolled Keywords:||human computation, crowdsourcing, max algorithms, worker models, vote aggregation, plurality voting|
|Related URLs:||Author Homepage||http://i.stanford.edu/~venetis/|
|Deposited By:||Petros Venetis|
|Deposited On:||07 Nov 2011 10:51|
|Last Modified:||18 Feb 2012 18:14|
Repository Staff Only: item control page