Stanford InfoLab Publication Server

Pay-As-You-Go Entity Resolution

Whang, Steven Euijong and Marmaros, David and Garcia-Molina, Hector (2012) Pay-As-You-Go Entity Resolution. IEEE Transactions on Knowledge and Data Engineering .


This is the latest version of this item.

PDF - Published Version


Entity resolution (ER) is the problem of identifying which records in a database refer to the same entity. In practice, many applications need to resolve large data sets efficiently, but do not require the ER result to be exact. For example, people data from the Web may simply be too large to completely resolve with a reasonable amount of work. As another example, real-time applications may not be able to tolerate any ER processing that takes longer than a certain amount of time. This paper investigates how we can maximize the progress of ER with a limited amount of work using ``hints,'' which give information on records that are likely to refer to the same real-world entity. A hint can be represented in various formats (e.g., a grouping of records based on their likelihood of matching), and ER can use this information as a guideline for which records to compare first. We introduce a family of techniques for constructing hints efficiently and techniques for using the hints to maximize the number of matching records identified using a limited amount of work. Using real data sets, we illustrate the potential gains of our pay-as-you-go approach compared to running ER without using hints.

Item Type:Article
ID Code:1022
Deposited By:Steven Whang
Deposited On:27 Jan 2012 18:02
Last Modified:28 Jan 2012 15:18

Available Versions of this Item

Download statistics

Repository Staff Only: item control page