ee465: introduction to digital image processing 1 what is in common?

EE465: Introduction to Digital Image Processing

1

What is in Common?


2

A BW Image and Its Inverse


3

Application (I): FAX Document


4

Application (II): Scanned Documents


5

Application (III): Biometrics


6

Application (IV): Image Matting

figure-ground segmentationin computer vision


7

Binary Image Processing Introduction Set theory review Morphological filtering operators

• Erosion and dilation• Opening and closing• Hit-or-miss • Boundary extraction• Region-filling• Thinning (for finding skeleton)

Other processing algorithms• Area calculation, finding connected components• Skeleton finding via distance transform


8

Binary Images

Images only consist of two colors (tones): white or black

Numerical example (image of a square block)

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 255 255 0 0 0 0 255 255 255 255 0 0 0 0 255 255 255 255 0 0 0 0 255 255 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0


9

Why are binary images special?

Since pixels are either white or black, the locations of white (black) pixels carry ALL information of binary imagesExample

0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 1

f(m,n)

L={(3,3),(3,4),(4,3),(4,4)}

location of white pixels

It is often more convenient to consider the set representation than the matrix representation for binary images

matrix representation set representation


10


• Erosion and dilation

• Opening and closing

• Hit-or-miss

• Boundary extraction

• Region-filling

• Thinning (for finding skeleton) Other processing algorithms

• Area calculation, finding connected components

• Skeleton finding via distance transform


11

Set Theory Review

}|{ AwwAc cBABwAwwBA },|{Think of sets A and B as the collections of spatial coordinates


12

Translation Operator

},|{)( AazawwA z

),( 21 zzz

Example


13

Reflection Operator

},|{ˆ BbbwwB

Example

B B̂


14


• Erosion and dilation

• Opening and closing

• Hit-or-miss

• Boundary extraction

• Region-filling

• Thinning (for finding skeleton) Other processing algorithms

• Area calculation, finding connected components

• Skeleton finding via distance transform


15

Structuring Element B

Definition: a set of local neighborhood with specified origin

Examples

origin

B1 B2

Note: different structuring element leads to different filtering result


16

mask B

X Y

XBBXBXYBb Xx

xb

Definition

Example

Dilation

})ˆ(|{ XBzBXY z

or


17

X

Xca

B={a,b,c}

BX

a=(0,0),b=(0,-1),c=(-1,0)

origin

Xcb

B

XccBX c

Illustration by Animation


18

mask B

X Y

Definition

Example

Erosion

Y=X B_ }:{}:{ cxx XBxXBx


19

mask B

XY

Illustration By Animation


20

(X B)c_ =XcB̂

Duality Property*

Proof:

(X B)c_={z | Bz A }c

= {z | Bz Ac = }c

= {z | Bz Ac }

=Xc B^


21

X

B

Xc

B̂

(X B)c_X B_

XcB̂

Example


22





23

(X B) B_ +Definition

mask B

X

_

+

BX

BX

Opening Operator

Example


24

Geometric Interpretation of Opening Operator


25

Definition (X B) B+ _

_

+

mask B

BX

X

BX

Closing Operator

Example


26

Geometric Interpretation of Closing Operator


27

Properties of Opening and Closing Operators*

Opening

Closing

XBX ●

●

●

BYBXYX BXBBX )(

BXX ●

●

●

BYBXYX BXBBX )(

BXBX CC ˆ)( BXBX CC ˆ)(


28





29

A Little Game of Matching

Templates

A

B


30

Illustration by a Simpler Case

Template B

How to find the match of A in Xusing a computer?

X

Hit: the southwest quadrant must be black in X

origin

Hit: the other three quadrants must be white in X


Hit: the other three quadrants must be black in Xc


31

Matching via Hit-or-Miss

Template B

origin


Hit: the other three quadrants must be black in Xc

Template B1

Template B2

X1=X B1_

X2 =Xc B2_

To satisfy both conditions, we need to take the intersection of X1 and X2


32

mask B1

(MATLAB function: bwhitmiss)

(X B1)(Xc B2)_ _X B=*

Hit Miss

mask B2

origin

mask B

x

x x

MATLAB

0 -1 -11 1 -10 1 0

Hit-or-Miss Operator

Definition

Structuring element example

Why ? _

Why complement?Why intersection?


33

X

Example 1

mask B1 mask B2

origin

Xc Xc B2X B1_ _

X B*


34

mask B1 mask B2 mask B MATLAB

-1 -1 -11 1 -11 1 -1

Example 2 origin

X Xc Xc B2X B1_ _

X B*Now, take the intersectionand note that only two pointsRemain (highlighted by red)


35





36

“BAD EXAMPLE”

Y=X-(X B)_

mask B

X X B_


37

X-(X B)_

mask B

X X B_

Boundary Extraction

Example

X=X-(X B)_Definition

X=(X B) – B?+

How about

cBABA Note that


38

XX

Image Example


39

Inner vs. Outer Boundary


40





41

P

X

Iterations:

Y0=P

Yk=(Yk-1B)Xc, k=1,2,3…mask B

Terminate when Yk=Yk-1,output YkX

Idea: recursively expand the region around P but stopthe expansion at the boundary of X

Region Filling

expansion stop at the boundary

Why dilation?Why intersection with Xc?


42

X Y0

P

Y0B Xc Y1

Y1B Y2Y2B Y3=Y2

Image Example


43

Additional Example

P


44





45

Thinning

thinning

Intuitively, thinning finds the skeleton of a binary image (you will learn adifferent way of finding skeleton by distance transform later)


46

Thinning Algorithm

Basic idea: Use Hit-or-Miss operator as a sifter

Use multiple masks to characterize different patterns

x

xB2

x x

B1

x

x

B3

x

x

B4 B6

x x

B5 B7 B8

x

x x

x

x

x

X0=X

Xk=(…( (Xk-1 B1) B2 … B8)

where X B=X – X B*Stop the iteration when Xk=Xk-1

Why eight different B’s?Why hit-or-miss?


47

Image Example


48

Image Example (Con’t)

Binary fingerprint image Skeleton image


49

Morphological Filtering: Skeletons*

X kB=(((X B) B) … B)_ _ _ _

k-fold

Define

Sk(A)=(A kB)-(A KB) B_ _and

Then the skeleton of an image A is given by

0

)()(k

k ASAS


50

Textbook Example*


51

Image Example*

Binary fingerprint image Skeleton image


52

Summary of Morphological Filtering

MATLAB codes

circshift(A,z)

fliplr(flipud(B))

~A or 1-A

A &~B

imdilate(A,B)

imerode(A,B)

imopen(A,B)

imclose(A,B)


53

Summary (Con’d)

bwhitmiss(A,B)

A&~(imerode(A,B))

region_fill.m

bwmorph(A,’thin’);


54

What is Common?


55

One-Minute Survey

If X={even integers}, Y={multiples of 6}, what is X-Y?

How long did you spend on CA#1? What do you think of the pace of this course so

far? What is the muddiest point in the last two

weeks’ lectures? Do you mind my posting the class grade using

the last 4-digit of your ID?


56





57

Building New Pieces

How to calculate the perimeter of a binary object A?• Find the boundary first (MATLAB: bwperim(A))

How to calculate the area of a binary object A?• MATLAB: bwarea or sum(sum(A))

• Complication: what if there are multiple binary objects?

(this relates to binary image analysis. MATLAB: bwlabel)

How to get rid of small holes of a binary object?• Recall region filling routine (MATLAB: bwareaopen)


58

Image Examples

A label2rgb(bwlabel(A)) bwareaopen(A,50)


59

Medial Axis Transform (a.k.a. Skeleton)

• What is medial axis?

• Examples

Suppose that a fire line propagates with constant speed from thecontour of a connected object towards its inside, then all those points lying in positions where at least two wave fronts of the fire line meet during the propagation will constitute a form of a skeleton


60

Distance Transform

http://demonstrations.wolfram.com/DistanceTransforms/


61

Skeleton Algorithm

• Distance transform: find the distance from thenearest boundary for each point

• Skeleton is the set of points whose distance from thenearest boundary is locally maximum

),(),(0 nmxnmx - initialization

- iteration }1),;,(:),(min{),(),( 1 jinmdjixnmxnmx kok

}1),;,(),,(),(:),{( jinmdjixnmxnm kk


62

1 1 1 1 11 1 1 1 11 1 1 1 11 1 1 1 11 1 1 1 1

1 1 1 1 11 2 2 2 11 2 2 2 11 2 2 2 11 1 1 1 1

1 1 1 1 11 2 2 2 11 2 3 2 11 2 2 2 11 1 1 1 1

1 1 2 2 3 2 21 1

x0(m,n) x1(m,n) x3(m,n) skeletonlocal maximum

• Example

Skeleton Algorithm (Cont’d)


63

Image Example

original skeleton


64

Binary Image Processing Summary

ee465: introduction to digital image processing 1 what is in common?

Documents

digital image processing

bw image

animation slide

inverse slide

black numerical example

distance transform slide

computer vision slide

fax document slide