chapter 11: image description and representationit.nrru.ac.th/krit/411304/chapter11.pdf · wood,...

Digital Image ProcessingDigital Image ProcessingDigital Image ProcessingChapter 11:

Digital Image ProcessingChapter 11:p

Image Description andRepresentation

pImage Description and

RepresentationRepresentationRepresentation

Image Representation and Description? Image Representation and Description? Objective:Objective:

To represent and describe information embedded inan image in other forms that are more suitable than thean image in other forms that are more suitable than the image itself.

fiBenefits:- Easier to understand

R i f f t t b d- Require fewer memory, faster to be processed- More “ready to be used”

What kind of information we can use?- Boundary, shapey, p- Region- Texture- Relation between regions

Shape Representation by Using Chain Codes Shape Representation by Using Chain Codes Why we focus on a boundary?Why we focus on a boundary?The boundary is a good representation of an object shapeand also requires a few memoryChain codes: represent an object boundary by a connected sequence of straight line segments of specified length

and also requires a few memory.

sequence of straight line segments of specified lengthand direction.

4 di i l4-directionalchain code

8-directionalchain code

(Images from Rafael C.Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Examples of Chain Codes Examples of Chain Codes

Object BoundaryObjecboundary

(resampling)

Boundaryvertices

4-directional 8-directionald ec ochain code

8 d ec ochain code

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

The First Difference of a Chain Codes The First Difference of a Chain Codes Problem of a chain code:Problem of a chain code:

a chain code sequence depends on a starting point.Solution: treat a chain code as a circular sequence and redefine thestarting point so that the resulting sequence of numbers forms an integer of minimum magnitude.

The first difference of a chain code: counting the number of directionchange (in counterclockwise) between 2 adjacent elements of the code.

Example:1

Example: - a chain code: 10103322

Chain code : The first difference

2 0- The first difference = 3133030- Treating a chain code as a

circular sequence we get

0 1 10 2 20 3 3

3circular sequence, we getthe first difference = 33133030

0 3 32 3 12 0 22 1 3 The first difference is rotational

invariant.

Polygon Approximation Polygon Approximation

Represent an object boundary by a polygon

Object boundary Minimum perimeterpolygonpolygon

Minimum perimeter polygon consists of line segments thatMinimum perimeter polygon consists of line segments that minimize distances between boundary pixels.


Polygon Approximation:Splitting Techniques Polygon Approximation:Splitting Techniques

1. Find the line joiningtwo extreme points

0 Object boundary0. Object boundary 2. Find the farthest pointsfrom the linefrom the line

3. Draw a polygon(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

DistanceDistance--VersusVersus--Angle Signatures Angle Signatures Represent an 2-D object boundary in term of a 1-D functionRepresent an 2 D object boundary in term of a 1 D function of radial distance with respect to .


Boundary Segments Boundary Segments Concept: Partitioning an object boundary by using verticesConcept: Partitioning an object boundary by using vertices of a convex hull.

Partitioned boundary

Convex hull (gray color)

Object boundary


Input : A set of points on a cornea boundary

Convex Hull Algorithm Convex Hull Algorithm Input : A set of points on a cornea boundaryOutput: A set of points on a boundary of a convex hull of a cornea1 Sort the points by x-coordinate to get a sequence p p … p1. Sort the points by x-coordinate to get a sequence p1, p2, … ,pnFor the upper side of a convex hull2. Put the points p and p in a list L with p as the first point2. Put the points p1 and p2 in a list Lupper with p1 as the first point3. For i = 3 to n4. Do append pi to L4. Do append pi to Lupper5. While Lupper contains more than 2 points and the last 3

points in L do not make a right turnp upper g6. Do delete the middle point of the last 3 points from Lupper

TurnRight

TurnRight

TurnLeftNOK!Right

OK! OK!

For the lower side of a convex hullConvex Hull Algorithm (cont.) Convex Hull Algorithm (cont.)

7. Put the points pn and pn-1 in a list Llower with pn as the first point8. For i = n-2 down to 18. For i n 2 down to 19. Do append pi to Llower10. While L contains more than 2 points and the last 3 points10. While Llower contains more than 2 points and the last 3 points

in Llower do not make a right turn11. Do delete the middle point of the last 3 points from L11. Do delete the middle point of the last 3 points from Llower

12. Remove the first and the last points from Llower

13 Append L to L resulting in the list L13. Append Llower to Lupper resulting in the list L14. Return L

TurnTurnRight

TurnRightOK!

TurnOK!

TurnLeftNOK!

Skeletons Skeletons Obtained from thinning or skeletonizing processesObtained from thinning or skeletonizing processes

Medial axes (dash lines)Medial axes (dash lines)


Thinning Algorithm Thinning Algorithm Concept: 1. Do not remove end pointsp p

2. Do not break connectivity3. Do not cause excessive erosion

Apply only to contour pixels: pixels “1” having at least one of its 8neighbor pixels valued “0”

iNeighborhoodarrangement

p2p9 p3

Notation:

arrangementfor the thinningalgorithm

p8

p7

p1 p4

p6 p5

=Let

E lgp7 p6 p5

Let 98321)( pppppN 00 1

Example

T(p1) = the number of transition 0-1 in the ordered sequence p2, p3, …

11

p1 00 1

, p8, p9, p2. N(p1) = 4T(p1) = 3

Thinning Algorithm (cont.) Thinning Algorithm (cont.)

Step 1. Mark pixels for deletion if the following conditions are true.a) b) T( ) 1

6)(2 1 pNp2p9 p3(Apply to all border pixels)b) T(p1) =1

c)d)

0642 ppp0ppp

p2p9 p3

p8

p7

p1 p4

p6 p5

(Apply to all border pixels)

d) 0864 ppp p7 p6 p5

Step 2. Delete marked pixels and go to Step 3.

Step 3. Mark pixels for deletion if the following conditions are true.a) 6)(2 1 pN (Apply to all border pixels)b) T(p1) =1c) 0842 ppp

0

(Apply to all border pixels)

d) 0862 ppp

Step 4. Delete marked pixels and repeat Step 1 until no changep p p p goccurs.

Example: Skeletons Obtained from the Thinning Alg. Example: Skeletons Obtained from the Thinning Alg.

Skeleton


Boundary Descriptors Boundary Descriptors 1 Simple boundary descriptors:1. Simple boundary descriptors:we can use

Length of the boundary- Length of the boundary- The size of smallest circle or box that can totallyenclosing the objectenclosing the object

2 Shape number2. Shape number

3 Fourier descriptor3. Fourier descriptor

4 Statistical moments4. Statistical moments

Shape Number Shape Number Shape number of the boundary definition: 1Shape number of the boundary definition:

the first difference of smallest magnitudeThe order n of the shape number: 2 0p

the number of digits in the sequence3


Shape Number (cont.) Shape Number (cont.)

Shape numbers of order S pe u be s o o de4, 6 and 8


Example: Shape Number Example: Shape Number

2. Find the smallest rectanglethat fits the shape

1. Original boundarythat fits the shape

Chain code: 0 0 0 0 3 0 0 3 2 2 3 2 2 2 1 2 1 1

First difference:First difference:3 0 0 0 3 1 0 3 3 0 1 3 0 0 3 1 3 0

Shape No4. Find the nearest Shape No.0 0 0 3 1 0 3 3 0 1 3 0 0 3 1 3 0 33. Create grid

4. Find the nearest Grid.


Fourier Descriptor Fourier Descriptor Fourier descriptor: view a coordinate (x,y) as a complex number p ( y) p(x = real part and y = imaginary part) then apply the Fourier transform to a sequence of boundary points.

)()()( kjykxks Let s(k) be a coordinate of a boundary point k :

1

/2)(1)(K

KukeksK

ua Fourier descriptor : 0

)()(kK

p

Reconstruction formula

1

/2)(1)(K

Kukeuaks

Reconstruction formula

0

)()(k

euaK

ks

d(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Boundarypoints

Example: Fourier Descriptor Example: Fourier Descriptor Examples of reconstruction from Fourier descriptorsExamples of reconstruction from Fourier descriptors

1

/2)(1)(ˆP

Kukeuaks

0

)()(k

euaK

ks

P is the number ofP is the number of Fourier coefficients used to reconstruct th b dthe boundary


Fourier Descriptor Properties Fourier Descriptor Properties

Some properties of Fourier descriptors


Statistical Moments Statistical Moments Definition: the nth moment Example of moment:

1

)()()(K

in

in rgmrr

Definition: the n moment Example of moment:The first moment = meanThe second moment = variance

0i

where

1

)(K

ii rgrm 0

)(i

ii g

Boundarysegment 1D graph

1. Convert a boundary segment into 1D graph2 View a 1D graph as a PDF function


2. View a 1D graph as a PDF function3. Compute the nth order moment of the graph

Regional Descriptors Regional Descriptors Purpose: to describe regions or “areas”Purpose: to describe regions or areas

1. Some simple regional descriptorsarea of the region- area of the region

- length of the boundary (perimeter) of the region- Compactness )(RAp

)()(

2 RPRAC

where A(R) and P(R) = area and perimeter of region RExample: a circle is the most compact shape with C = 1/4

2 T l i l D i t2. Topological Descriptors3. Texture4 Moments of 2D Functions4. Moments of 2D Functions

Example: Regional Descriptors Example: Regional Descriptors

White pixels represent “light of the cities”

% f hit i l% of white pixelsRegion no. compared to the

total white pixels

1 20.4%2 64.0%3 4.9%4 10.7%

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.Infrared image of America at night

Topological Descriptors Topological Descriptors Use to describe holes and connected components of the regionUse to describe holes and connected components of the region

Euler number (E):

HCE

C = the number of connected components

H = the number of holes


Topological Descriptors (cont.) Topological Descriptors (cont.)

E = -1

E = 0 Euler Formula

EHCFQV

V = the number of verticesh b f dQ = the number of edges

F = the number of faces


E = -2

Example: Topological Descriptors Example: Topological Descriptors

Original image:Infrared imageOf Washington

After intensityThresholdingg

D.C. area (1591 connectedcomponents with 39 holes)Euler no. = 1552

The largestconnected After thinning

area (8479 Pixels)

(Hudson river)(Hudson river)


Texture Descriptors Texture Descriptors Purpose: to describe “texture” of the region.p gExamples: optical microscope images:

B C

A

B C

Superconductor Cholesterol Microprocessor


p(smooth texture) (coarse texture)

p(regular texture)

Statistical Approaches for Texture Descriptors Statistical Approaches for Texture Descriptors We can use statistical moments computed from an image histogram:We can use statistical moments computed from an image histogram:

1

)()()(K

in

in zpmzzz = intensityp(z) = PDF or histogram of z

0i

where

1

)(K

ii zpzm

p(z) PDF or histogram of z

0

)(i

ii p

Example: The 2nd moment = variance measure “smoothness”Example: The 2 moment variance measure smoothnessThe 3rd moment measure “skewness”The 4th moment measure “uniformity” (flatness)

AB


C

Fourier Approach for Texture Descriptor Fourier Approach for Texture Descriptor

C t t 2D t i t 1D h

F i

Concept: convert 2D spectrum into 1D graphs

Divide into areasby angles

Originalimage

Fouriercoefficient

imageFFT2D

+FFTSHIFT image

S ll i lDivide into areas Sum all pixelsin each area

Divide into areasby radius

0

)()(R

rSS 1r

r

Sum all pixelsin each area

0

)()( rSrS

Fourier Approach for Texture Descriptor Fourier Approach for Texture Descriptor

Originalimage

2D Spectrum(Fourier Tr.)( )

S(r) S()S( ) S()

Anotherimage

Another S()


Moments of TwoMoments of Two--D Functions D Functions The moment of order p + q

x y

qppq yxfyxm ),(

00

10

mmx

00

01

mmy

The moment of order p + q

00 00

The central moments of order p + q

x y

qppq yxfyyxx ),()()(

0000 m 01001

1011011111 mymmxm

102020 mxm 010202 mym 102020 mxm 010202 mym

012

20112121 22 mxmymxm 102

203030 23 mxmxm

102

02111212 22 mymxmym 012

020303 23 mymym

Invariant Moments of TwoInvariant Moments of Two--D Functions D Functions

Th li d t l t f d +

pq where 1

qp

The normalized central moments of order p + q

00pq where 1

2

Invariant moments: independent of rotation translation scalingInvariant moments: independent of rotation, translation, scaling,and reflection

22 4 02201 1102202 4

203212

12303 33 203212

12304

22

20321

21230123012305

33

33

20321

2123003210321 33

20321

2123002206

0321123011

0321123002206

4

Example: Invariant Moments of TwoExample: Invariant Moments of Two--D Functions D Functions

3 Mi d1. Original image 2. Half size 3. Mirrored

4. Rotated 2 degree 5. Rotated 45 degree


Example: Invariant Moments of TwoExample: Invariant Moments of Two--D Functions D Functions

Invariant moments of images in the previous slide

Invariant moments are independent of rotation, translation,scaling, and reflection


Principal Components for Description Principal Components for Description Purpose: to reduce dimensionality of a vector image while maintainingPurpose: to reduce dimensionality of a vector image while maintaining

information as much as possible.

Let Tnxxx ]...[ 21x

Mean: K

kx KE 1}{ xxm

kK 1

C i t i K

TTTE 1}))({(CCovariance matrix

k

Txx

Tkk

Txxx K

E1

1}))({( mmxxmxmxC

Hotelling transformation Hotelling transformation

)(Let )( xmxAy

Where A is created from eigenvectors of C as followsWhere A is created from eigenvectors of Cx as followsRow 1 contain the 1st eigenvector with the largest eigenvalue.Row 2 contain the 2nd eigenvector with the 2nd largest eigenvalue.

00Then we get

….

1

1

0...00...0

C0}{ ym Ey

and

1......0............

yCT

xy AACC and

Then elements of are uncorrelated. The component of y with the largest is called the principal

)( xmxAy component of y with the largest is called the principal component.

Eigenvector and Eigenvalue Eigenvector and Eigenvalue

Eigenvector and eigenvalue of Matrix C are defined as

Let C be a matrix of size NxN and e be a vector of size Nx1Let C be a matrix of size NxN and e be a vector of size Nx1.If

eCe eCe

Where is a constantWhere is a constant

We call e as an eigenvector and as eigenvalue of CWe call e as an eigenvector and as eigenvalue of C

Example: Principal ComponentsExample: Principal Components

6 spectral imagesfrom an airborneScanner.


Example: Principal Components (cont.)Example: Principal Components (cont.)

Component 1 32102 931.43 118.54 83 884 83.885 64.006 13 40


6 13.40

Example: Principal Components (cont.)Example: Principal Components (cont.)

Original image After Hotelling transform

Principal Components for Description Principal Components for Description


Relational Descriptors Relational Descriptors


Structural Approach for Texture Descriptor Structural Approach for Texture Descriptor


chapter 11: image description and representationit.nrru.ac.th/krit/411304/chapter11.pdf · wood,...

Documents