Disambiguation Algorithm for

People Search on the Web

Dmitri V. Kalashnikov, Sharad Mehrotra, Zhaoqi Chen, Rabia Nuray-Turan, Naveen Ashish

For questions visit: http://www.ics.uci.edu/~dvk

Computer Science DepartmentUniversity of California, Irvine

2

Entity (People) Search

Person1 Person2

Person3

Unknown beforehand

Top-K Webpages

3

Standard Approach to Entity Resolution

"J. Smith"

f2

f3

?

?

?

js@mit.edu

Yf2

f3

sm@yahoo.com?

X

Traditional MethodsFeatures and Context

"Jane Smith"

4

Key Observation: More Info is Available

A "nice" regular DatabaseJane Smith

John Smith

J. Smith

=

5

RelDC Framework

f1

f2

f3

?

?

?

f4

Y

f1

f2

f3

f4?

X

Traditional Methods

+ X Y

A

B C

D

E F

Relationship Analysis

ARG

RelDC Framework

features and context

Relationship-based Data Cleaning

6

Where is the Graph here? Use Extraction!

Andrew McCallum

Lise Getoor

Ben Taskar University of Massachusetts

Amherst

Charles Sutton

cs.umass.edu

umass.edu

mccallum@cs.umass.edu

cs.umass.edu/~mccallum

Amherst, MA

People Organizations

Locations

Hyperlinks/Email

cs.umass.edu/~mccallum/bio.html

7

Overall Algorithm Overview

1. User Input. A user submits a query to the middleware via a web-based interface.

2. Web page Retrieval. The middleware queries a search engine’s API, gets top-K Web pages.

3. Preprocessing. The retrieved Web pages are preprocessed:a) TF/IDF. Preprocessing steps for computing TF/IDF are carried out.b) Ontology. Ontologies are used to enrich the Webpage content.c) Extraction. Named entities, and web related information is extracted from the

Webpages.4. Graph Creation. The Entity-Relationship Graph is generated5. Enhanced TF/IDF. Ontology-enhanced TF/IDF values are computed6. Clustering. Correlation clustering is applied7. Cluster Processing. Each resulting cluster is then processed as follows:

a) Sketches. A set of keywords that represent the web pages within a cluster is computed for each cluster. The goal is that the user should be able to find the person of interest by looking at the sketch.

b) Cluster Ranking. All cluster are ranked by a choosing criteria to be presented in a certain order to the user

c) Web page Ranking. Once the user hones in on a particular cluster, the Web pages in this cluster are presented in a certain order, computed on this step.

8. Visualization of Results. The results are presented to the user in the form of clusters (and their sketches) corresponding to namesakes and which can be explored further.

8

Correlation Clustering

• In CC, each pair of nodes (u,v) is labeled– with “+” or “-” edge– labeling is done according to a similarity function s(u,v)

• Similarity function s(u,v)– if s(u,v) believes u and v are similar, then label “+”– else label “-”– s(u,v) is typically trained from past data

• Clustering– looks at edges– tries to minimize disagreement

– disagreement for element x placed in cluster C, is a number of “-” edges that connect x and other elements in C

9

Similarity Function

• Connection strength between u and v:

– where ck – the number of u-v paths of type k

– and wk – the weigh of u-v paths of type k

• Similarity s(u,v) is a combination

10

Training s(u,v) on pre-labeled data

11

Experiments: Quality of Disambiguation

By Artiles, et al. in SIGIR’05

By Bekkerman & McCallum in WWW’05

12

Experiments: Effect on Search