optical character recognition: using the ullman algorithm for graphical matching iddo aviram

Optical Character Recognition:Using the Ullman Algorithm for Graphical Matching

Iddo Aviram

OCR- a Brief Review

• Optical character recognition, usually abbreviated to OCR, is the mechanical or electronic translation of scanned images of handwritten, typewritten or printed text into machine-encoded text.

• OCR is a task, and not a mathematically defined problem.

OCR- a Brief Review

• People are using many disciplines for OCR.

• We will show just a simple, not representative, approach to deal partly with the OCR task.

Fourier Transforms

Pattern Matching

Machine Learning

Differential GeometryComputer Vision

Neural Networks

Expert Systems

Optimization Problems

Topology

Decision Making

OCR- a Brief Review• The task can be very hard, and state-of-the-art algorithms

might be not good enough for some practical purposes. In several cases, however, OCR tools can perform well and be useful.

Harder EasierHandwritten Printed

Cursive Block letters

Free handwritten Scribe script

Offline Online

Connected writing Non-connected writing

Degraded manuscripts Well-preserved manuscripts

Non restricted writing Restricted writing

OCR- a Brief Review

• The human brain does amazingly well with OCR tasks, so usually the computer results are evaluated by a comparison with a manually created ground truth data.

• However, sometimes even humans are not capable of recognition.

OCR- a Brief Review

• Can you read these scripts?

: תקווה בפתח נדלן: למעלה)

,yad1.co.ilמתוך 2012

למטה:" " חבצלת ה מתוך

1912)

OCR- a Brief Review

• Can you read this script?

מוקדמת גרסה + " כאב" שיר ל

," רבים" מים שיר , אריאל מאיר- ה שנות 70סוף

OCR- a Brief Review

• Can you read this script?

. ? : את“ תן �ועת לקוס � והברכתך �את הש�לם לבלבל אמר למלך אמר ] [ ] [ חמר ] [ י פן קוס בח מז על אל ז ע והרם אחאמה עמד אשר ה�אכל

ה�אכל.”

חרס על כתובת- ) אוסטרקון)עוזה חורבת

הברזל , IIתקופת- ה לפני 7המאההספירה

חרס על דיוהעתיקות רשות

• Using graphical tools for object recognition.• A possible scheme:– Binarization– Segmentation by connected components – Thinning– Graphical modeling– Graphical matching– Rule-Based Selection

OCR- Motivation for Graphical Matching

• Binarization:


• Segmentation-> Thinning-> Graphical modeling:


• Given an historical manuscript, a blessing of Brit Milah:


• We’re interested in finding the occurrences of the letter Mem (not final):


• By sub-graph matching we can find candidates:


Graphical modeling

Graphical matching

• Given two graphs H and G as input, the problem is whether H has a subgraph that is isomorphic to G.

Subgraph Isomorphism Problem

3

1

2 4

2

1

3

H

G

• In this example the answer is ‘yes’ since there’s an isomorphic correspondence:

1G-1H,2G-3H,3G-2H.(There are additional isomorphic correspondences).

• Graph isomorphism – Graphs G(VG,EG) and H(VH,EH) are isomorphic if

|VG|=|VH| and there is an invertible function F from VG to VH such that for all nodes u and v in VG, (u,v) E∈ G if and only if (F(u),F(v)) E∈ H.

– Such a function F is said to be an isomorphic correspondence.


• The subgraph problem is NP-complete.• There is a very simple reduction:CLIQUE ≤P Subgraph Isomorphism

• However, for many specific types of practical problems (even with ‘big’ inputs), algorithms do answer fast.


• An Algorithm for Subgraph Isomorphism, J. R. Ullmann, Journal of the ACM, 1976.

• Although old, this algorithm is still very popular and having good results in practice.

The Ullman Algorithm

• There are algebraic formulations for graph isomorphism and subgraph isomorphism, that we will take use of.

• The adjacency matrix AH of a graph H would be:


• We will use the notion of a permutation matrix.• Any permutation matrix is equivalent to an isomorphic

correspondence.


M’=- - - -

Isomorphic Correspondence Permutation Matrix

F= F~M’

• Two graphs and are isomorphic with a correspondence F is similar to , and the similarity matrix is M’~F.


F= 𝐴𝐻 2=𝑀 ′ 𝐴𝐻𝑀 ′− 1

Isomorphic Correspondence Permutation Matrix~

- - - -

M’=

F~M’

Isomorphism criterion:

iff is isomorphic to H, with a correspondence F~M’.

𝐴𝐻 2=𝑀 ′ (𝑀 ′ 𝐴𝐻)𝑇

• We can develop this equation that defines an isomorphism:


Since M’ is an orthonormal matrix, thus =I

Since is a symmetric matrix

Isomorphism criterion:

iff is isomorphic to H, with a correspondence F~M’.

• In a similar fashion (without proof) we have an algebraic criterion for a subgraph isomorphism.


M=’1G-1H

2G-3H

3G-2H

4G-φ

Isomorphic Correspondence Permutation Matrix~

F= Subgraph isomorphism criterion:

𝐴𝐺=𝑀 ′ (𝑀 ′ 𝐴𝐻)𝑇iff G is subgraph isomorphic to H, with a correspondence F~rectangular M’.

• We have a graph G and a graph H, and we want to know if G is subgraph isomorphic to H .

• So, We’ll search for a permutation matrix M* of size |x || that satisfies the subgraph isomorphism criterion.• We will enumerate over candidate permutation matrices of the

same size, denoting a candidate by M’, from a set of candidates that satisfies:

(The set of all M*-s) (The set of all M’-s) . During the enumeration, we check the isomorphism criterion over each candidate. If a candidate satisfies the criterion, we will return ‘yes’. If we would not find such a candidate, we will return ‘no’.


The Ullman Algorithm• Ullmann’s algorithm I

– Construction of another matrix M(0) with the same size of the M’-s:

– Generation of all M’-s by setting to 0 all but one 1 in each row of M(0)

– A subgraph isomorphism has been found if M implies: .

}1,0{,otherwise0

)deg()deg(if1,

)0(,

jiGiHj

ji mVV

m

)1()1( ,, jijiG pa

3

1

2 4

2

1

3

H

G

011100100

0010001011010010

G

H

A

A

001011111111

0M

The Ullman Algorithm• Ullmann’s algorithm I

– Example

001011111111

001011110001

001011110010

001011110100

001011111000

001001000001

001010000001

001000010100

001010000100

001000011000

001001001000

1

3

2

4 1

3

3

2 2

3

1

4 1

3

1

2

1

2

3

4

2

3

3

1 1

3

2

1

011100100

)'(' THAMMP

011100100

with compared GA

1

3

2

Inner Nodes – M-s

Root - M(0)

Leaves – M’-s

The Ullman Algorithm• Ullman’s algorithm II

– Construction of another matrix M(0) with the same size of the M’-s:

– Generation of all M‘-s by setting to 0 all but one 1 in each row of M(0) . However, in this version, we will also prune all inner nodes M-s that have at least one 1 entry that doesn‘t comply with the refinement rule (to be defined). We are guaranteed to end up with the right answer since we still hold:

(The set of all M*-s) (The set of all M’-s) – A subgraph isomorphism has been found if there is M‘ that satisfies

.

}1,0{,otherwise0

)deg()deg(if1,

)0(,

jiGiHj

ji mVV

m

)1()1( ,, jijiG pa

The Ullman Algorithm• Ullmann’s refinement rule for prunning the search tree:

• Observation:• If a vertex of G, , corresponds to a vertex of H, , then for each

adjacent vertex of in G, denoted , there must be a vertex in H, denoted , in H that holds:

• A. is adjacent to in H• B. corresponds to

𝑣𝐻

𝑣𝐴𝐻

𝑣𝐺

𝑣𝐴𝐺

The Ullman Algorithm• Algebraic notation:

• For all mi,j = 1 (that is already fixed):

• Any inner node M that does not satisfy this rule is prunned, because all of its decendants are not M*-s.

optical character recognition: using the ullman algorithm for graphical matching iddo aviram

Documents