Stanford InfoLab Publication Server

Entity Resolution with Iterative Blocking

Whang, Steven Euijong and Menestrina, David and Koutrika, Georgia and Theobald, Martin and Garcia-Molina, Hector (2008) Entity Resolution with Iterative Blocking. Technical Report. Stanford.

WarningThere is a more recent version of this item available.



Entity Resolution (ER) is the problem of identifying which records in a database refer to the same real-world entity. An exhaustive ER process involves computing the similarities between pairs of records, which can be very expensive for large datasets. Various blocking techniques can be used to enhance the performance of ER by dividing the records into blocks in multiple ways and only comparing records within the same block. However, most blocking techniques process blocks separately and do not exploit the results of other blocks. In this paper, we propose an {\em iterative blocking framework} where the ER results of blocks are reflected to subsequently processed blocks. Blocks are now iteratively processed until no block contains any more matching records. Compared to simple blocking, iterative blocking may achieve higher accuracy because reflecting the ER results of blocks to other blocks may generate additional record matches. Iterative blocking may also be more efficient because processing a block now saves the processing time for other blocks. We implement a scalable iterative blocking system and demonstrate that iterative blocking is more accurate and efficient than blocking, especially for large datasets.

Item Type:Techreport (Technical Report)
Uncontrolled Keywords:entity resolution, blocking, iterative blocking
Subjects:Computer Science > Data Mining
Projects:Information Integration
Related URLs:Project Homepage
ID Code:835
Deposited By:Import Account
Deposited On:21 Jun 2008 17:00
Last Modified:09 Apr 2009 00:15

Available Versions of this Item

Download statistics

Repository Staff Only: item control page