Waldo: An Adaptive Human Interface for Crowd Entity Resolution

Abstract

In Entity Resolution, the objective is to find which records of a dataset refer to the same real-world entity. Crowd Entity Resolution uses humans, in addition to machine algorithms, to improve the quality of the outcome. We study an approach that combines two common interfaces for human tasks in Crowd Entity Resolution, taking into account some key observations about the advantages and disadvantages of the two interfaces. We give a formal definition to the problem of human tasks' selection and we derive algorithms with strong optimality guarantees. Our experiments with three real-world datasets show that our approach gives an improvement of 50% to 300% in the crowd cost to resolve a dataset, compared to using only tasks of the same interface.