ENTITY EXTRACTION: RULE-BASED METHODS

“I’m booked on the train leaving from Paris at 6 hours 31”

Rule: Location(Token string = “from”)({DictionaryLookup=location}):loc -> Location=:loc

Extraction through Rules Rules are useful for:

Creating wrappersExtracting information from controlled and well-behaved

data

“My email address is jane@byu.edu”

“For information you should call 801-111-2222”

Email address

Phone number

Rule-Based System: . Collection of

RulesPolicies to control the firing of multiple rules

Rule Representation Diverse rule-based systems employ

distinct formats for rule representation CSPL: Common Pattern Specification

Language Rapier: Robust Automated Production of

I.E. Rules WHISK: supervised algorithm to learn

regular expressions Avatar: SQL expressions

Form of a Basic Rule Contextual Pattern -> Action

1 or more labeled patterns capturing properties of 1 or more entities and the context in which they appear in the text Examples:

• Assigning entity label • Inserting a start/end of

an entity tag at a position

• Assigning multiple entity tags

Regular expressions defined over features of tokens in the text and an optional label

Tagging actions

Any property of the token or the context or the documents in which the token appears

Features of Tokens A token is associated with a bag of

features obtained through 1 or more criteria:

String representing the token

Orthography type of the token

Part of Speech of the token

List of dictionaries in which token s appear

Annotations attached by previous processing steps

<location> ….. </location>Heat water in a large

vessel

V N P DET ADJ N

Kitchen Jones & Jones

Locations:

RomeParis

Greece

Rules to Identify a Single Entity Patterns followed by entity-recognizing rules

An optional pattern to capture the context before the start of an entity

A pattern matching the tokens in the entity An optional pattern to capture the context after

the end of the entity

Example: Identify person names of the form “Dr. Yair Weiss”

{Orthography type = capitalized word} {2} )

({DictionaryLookup = Titles} {String = “.”} -> Person names

Rules to Identify a Single Entity Examples:

Rules for identifying company names in GATE, a popular entity recognition system

Rules to Mark Entity Boundaries Entity boundaries are useful to mark long

entities Separate rules are defined to mark the

start/end of entity boundary Each rule leads to the insertion of a SGML tag in

the text Example:

Insert <journal> tag to mark the start of a journal name in a citation record({String=“to”} {String=“appear”} {String=“in”}):jstart

({Orthography type = Capitalized word} {2-5}) -> insert <journal> after jstart

Rules for Multiple Entities Rules for multiple-entity recognition

Regular expression with multiple slots, each representing a different entity, to simultaneously identify more than one entity

Useful to extract information from structured records Medical records, classified ads, etc.

Example: Extract the number of bedrooms and rent from an

apt. rental ad({Orthography type = Digit}): Bedroom ({String=“BR”}) ({})* {String=“$”}) ({Orthography type = Number} ):Price -> Number of Bedrooms =: Bedroom, Rent =: Price

Organizing Collections of Rules Rule-based systems consist of very large

collection of rules Problem

Solution

How

Spans demarcated by different rules overlap, leading to conflicting actions

Component to organize rules and control de order in which they are applied to eliminate/resolve conflict

Use of heuristics and special –case handling, since rule-managing is a nonstandardized and custom-tuned part of rule-based system

Resolving Rule Conflicts Use of special/custom policies

Sample policies Prefer rules that mark larger spans of text as an

entity type Merge spans of text that overlap, only if the

action portion of the two applied rules is the same

Popular strategy since it allows flexibility in defining rules

Resolving Rule Conflicts Arrange rules as ordered set

Prioritize the order on all the rules and favor the one with higher priority Priority of a rule is fixed by some function of

the precision and coverage of the rule of the training data

“It is an open question whether a good rule-based theory should consist of rules that cover many

examples at the expense of a certain number of misclassifications or whether one should prefer rules that cover only few examples, but appear to be more

precise” (Fürnkranz, 2003)

Resolving Rule Conflicts Based on complete order

A later rule can be defined on actions of earlier rules

Example: Insert an end tag on the results of an earlier

rule used for inserting a start tag Since R1 has precedence, R2

can assume that <journal> can be used as part of the rule

Resolving Rule Conflicts Finite State Machines

A full automata is explicitly defined to control the exact sequence in which rules are applied Nodes (entities) are connected via directed edges

Each edge is associated with a rule on the input tokens that must be satisfied for the edge to be taken

Each rule correspond to a path in the FST There is no ambiguity about the order in which rules

are applied, as long as there is a unique path from the start to sink state for each sequence of tokens

How Rules are Created A typical entity extraction system

depends on a finely tuned set of rules

Rules manually coded by domain experts

Rules automatically learnt from labeled examples of entities in unstructured text

Rule Learning Algorithm Create a set of rules R1, R2, … Rk such

that the action of each rule either Identifies a single entity Marks entity boundaries Identifies multiple entities

From a training set consisting of Unstructured set of documents where all

the occurrences of entities are marked correctly

Rule Learning Algorithm Goal

Cover all the segments that contain an annotation by 1 or more rules

Ensure that the precision of each rule is high

Coverage of a rule R, i.e. S(R), is the fraction of data segments matched by R in the training documents

Precision of R is the ratio between S’(R), the subset of segments covered by R for which the action specified by R is correct, and S(R)

The overall set of rules must have good recall and precision on new documents

Rule Learning Algorithm Generalizability of learnt rules is required

Define the smallest set of rules that cover the maximum number of training cases with high precision Finding the optimal size for a rule set is

intractable Rule-learning algorithms follow a greedy hill

climbing strategy for learning one rule at the time

Rule Learning Algorithm The main challenge is to create a new

rule that achieves high overall coverage and has high precision These can be achieved using heuristics or

existing strategies, classified as

Bottom-up approach

Top-down approach

Specific Rule

Generalized Rule

General Rule

Specialized Rule

Bottom-Up Bottom-up rule formation

Start with a rule with minimal coverage but 100% precision

Gradually make rule more general to increase coverage, even if some precision is lost

Example Rule learning using (LP)2 for each tag type

Bottom-Up Creation of a seed rule

Example Seed rule to insert tag T=<PER> before a

position pstart

({String =“According”} {String =“to”}):pstart { String=“Robert”} {String =“Callahan”} -> insert <PER> at :pstart

Bottom-Up Generalizing seed rules

Example Replace or drop a token by a more general

feature token({String =“According”} {String =“to”}):pstart { String=“Robert”} {String =“Callahan”} -> insert <PER> at :pstart

({String =“According”} {String =“to”}):pstart {Orthography type =“Capitalized word”} {Orthography type =“Capitalized word”} -> insert <PER> after :pstart({DictioinaryLookup = Person}):pstart ({DictionaryLookup = Person}) -> insert <PER> before :pstart

Bottom-Up Generalizations retained starting from a

single seed rule Top-K rules are selected sequentially in

decreasing order of precision over uncovered instances (LP)2 also considers a number of measure of

quality rules, such as Precision Overall coverage Coverage of instances not covered by other rules

Top-Down Top-down rule formation

Start with a rule that covers all possible instances, i.e., 100% coverage and poor precision

Specialize rule to increase precision Select the set k of the most precise rules

Example Rule learning using (LP)2 for each tag typeUser-provided

threshold for the coverage of each rule

Rule Learning Algorithm Problem

Due to the limited availability of labeled data, purely automated data-driven method for rule induction are not adequate

Labeled Data

Rules

Solution Hybrid of automated and manual methods

to improve rule-based systems

Summary

Rule-based methods for entity extraction

Conflicts need to be resolved

“I’m booked on the train leaving from Paris at 6 hours 31”

Rule: Location(Token string = “from”)({DictionaryLookup=location}):loc -> Location=:loc

How sets of rules are created