digital image processing p. 1 edge models. p. 2 edges ... · 1 digital image processing lecture 9...

13
1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildreth edge detector Canny’s edge detector Edge linking using gradient information (Simple case.) using polygonial fit Hough transform Detection of lines Detection of circles Fourier Descriptors Find rotation using eigenvectors Gonzales & Woods: Chapter 10, pp. 711-742, Chapter 11, pp. 835-839,,861-868 Maria Magnusson, CVL, Institutionen för Systemteknik, Tekniska Högskolan, Linköpings Universitet p. 1 Edge models. Edges found in a real CT-image. Fig. 10.9 Fig. 10.8 step ramp roof p. 2 Noise increase after derivative Some low-pass filtering (smoothing) is often a good idea when computing derivatives! Use e.g. the Sobel filters. Fig. 10.11 Col 1: original Col 2: 1:st der. Col 3: 2:nd der. p. 3 The image gradient vector , ൌ mag (10-17) , ൌ tan (10-18) , ൌ arg , ∇ ≡ grad (10-16) Fig. 10.12 The edge direction is orthogonal to the gradient direction To get perfectly correct M and , g x and g y must be rotation invariant p. 4 1 2 3 4

Upload: others

Post on 04-Aug-2020

15 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

1

Digital Image Processing Lecture 9 Edge detection

The Marr-Hildreth edge detector Canny’s edge detector

Edge linking using gradient information (Simple case.) using polygonial fit

Hough transform Detection of lines Detection of circles

Fourier Descriptors Find rotation using eigenvectors Gonzales & Woods:

Chapter 10, pp. 711-742, Chapter 11, pp. 835-839,,861-868

Maria Magnusson, CVL, Institutionen för Systemteknik, Tekniska Högskolan, Linköpings Universitet

p. 1 Edge models.Edges found in a real CT-image.

Fig. 10.9

Fig. 10.8

step ramp roof

p. 2

Noise increase after derivative

Some low-pass filtering (smoothing) is often a good idea

when computing derivatives!

→Use e.g. the Sobel

filters.

Fig. 10.11Col 1:

originalCol 2:

1:st der.Col 3:

2:nd der.

p. 3

The image gradient vector

𝑀 𝑥, 𝑦 mag ∇𝑓 𝑔 𝑔 (10-17)

𝛼 𝑥, 𝑦 tan 𝑔 𝑔⁄ (10-18)𝛼 𝑥, 𝑦 arg 𝑔 , 𝑔

∇𝑓 ≡ grad 𝑓 ≡𝑔𝑔

𝜕𝑓𝜕𝑥𝜕𝑓𝜕𝑦

𝜕𝜕𝑥 ∗ 𝑓

𝜕𝜕𝑦 ∗ 𝑓

(10-16)

Fig. 10.12

The edge direction is

orthogonal to the gradient

direction

To get perfectly correct M and ,gx and gy must

be rotation invariant

p. 4

1 2

3 4

Page 2: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

2

Various gradient operators. Some you already know. Some are diagonal.

Fig. 10.14

Fig. 10.13

Do you remember

what constitutes a good edge detector?

1) Rotational invariance2) Sufficient smoothing

p. 5

To get perfectly correct M and ,gx and gy must be rotation invariant

𝐻 𝑢, 𝑣 𝑗2𝜋𝑢𝐻 𝜌

ℎ 𝑥, 𝑦𝜕

𝜕𝑥 ℎ 𝑟

To check the rotation invariance of a derivative filter hx, you can take the Fourier transform and divide by j2u, it gives H0, see 2, 3 and 4 slides forward.

Rotational symmetric functions. Not in G&W!Also lecture 4

The derivative corresponds to j2u. H0 gives rotational symmetric low-pass filtering.

p. 6

𝑓 𝑥, 𝑦

withrotationinvariantopera-tors

with-out

rotation invariantopera-tors

Rotation invariance, example

∇𝑓 𝑥, 𝑦 ∇𝑓 𝑥, 𝑦

Lab1: comparison sobel and simple

Lecture 4: It is desirable that two derivative filters gx and gy give a magnitude of the gradient (fx2+fy2)1/2 that is not dependent on rotation. Then the two derivative filters are rotation invariant.

p. 7

simplex0 0 01 0 10 0 0

/2

simpley0 1 00 0 00 1 0

/2

This simple pair is not especially rotation invariant

Take the Fourier transform and divide by j2u.

p. 8

5 6

7 8

Page 3: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

3

sobely1 2 1

0 0 01 2 1

/8

The Sobel pair is rather rotation invariant

Take the Fourier transform and divide by j2u.

sobelx1 0 12 0 21 0 1

/8

p. 9 Sufficient smoothing(low pass filtering) The sobel pair already contains some smoothing If more smoothing is desired: Take the derivative

of a Gaussian function (with the desired amount of smoothing), sample & truncate. This will also give good rotation invariance.

If more smoothing is desired: Convolve the image with a low-pass filter (i.e. Gaussian with the desired amount of smoothing) before applying the derivative filters (i.e. sobel).

p. 10

𝑔

Edge operations (based on sobel)

orig

inal

𝑔

See

also

lect

.3|g

x|+

|gy|

Fig. 10.16

p. 11

𝑔 ,  𝑔 ,  𝑔 𝑔 𝑔 𝑔  ? error in G&W

Edge operations (based on sobel), orig. image smoothed

Fig. 10.17

|gx|

+|g

y|

Smoothing removes noise and undesired details!

p. 12

𝑔or

igin

al

𝑔

9 10

11 12

Page 4: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

4

The Laplacian of Gaussian (LoG) filter

𝐺 𝑥, 𝑦 𝑒 ⁄

(10.2-21)(10.2-22)

Filters of arbitrary size can be received by sampling (10.2-22) and scaling the coefficients so that they sum to 0.Rule of thumb: Set the size of the discrete LoG filter to ≈6σ

(10-27)

∇ 𝐺 𝑥, 𝑦𝜕 𝐺 𝑥, 𝑦

𝜕𝑥𝜕 𝐺 𝑥, 𝑦

𝜕𝑦 (10-28)

It should be -1!

Fig. 10.21

-1ꞏ -1ꞏ

p. 13 The Difference of Gaussian (DoG) filterThe popular DoG filter can replace a LoG filter!

Fig. 10.23

LoG: solidDoG: dotted σ1: σ2 =1.75:1

DoG 𝑥, 𝑦1

2𝜋𝜎𝑒

12𝜋𝜎

𝑒

(10-32)

(10-33)𝜎𝜎 𝜎

𝜎 𝜎ln

𝜎𝜎

The human vision systemseems to contain DoG filters

with σ1: σ2 =1.75:1

σ1: σ2 =1.6:1

p. 14

-1ꞏDoG -1ꞏDoG

The Marr-Hildreth edge detector Var 1a) Apply a LoG or a DoG to the input image. Var 1b) Smooth the input image with a Gaussian

filter. Apply a simple negative laplacian filter.

2) Find the zero-crossings. If a) and b) below are true, the pixel is an edge pixel! a) At least two opposite pixels in a 3x3 neighborhood must

have different sign. b) Their absolute difference must also exceed a threshold

value.

𝑓 𝑥, 𝑦 ∇ 𝐺 𝑥, 𝑦 ∗ 𝑓 𝑥, 𝑦 ∇ 𝐺 𝑥, 𝑦 ∗ 𝑓 𝑥, 𝑦

(10-30), (10-31)

p. 15

The Canny edge detector 1) Low error rate. All edges should be found,

and there should be no spurious responses. 2) Edge points should be well localized. The

edges must be as close as possible to true edges. 3) Single edge point response. The detector

should not identify multiple edge pixels where only a single edge point exist

Lab 7

p. 16

13 14

15 16

Page 5: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

5

The Canny edge detector In 1D, Canny’s edge

detector identify an edge pixel as the position of the maximum of the absolute value of the 1:st derivative.

In 1D, there is also the zero-crossing of the 2:nd derivative.

In 2D, Canny’s edge detector identify an edge pixel as the position of the maximum of the magnitude of the gradient.

Edge location

p. 17

𝜕𝑓 𝑥𝜕𝑥

𝜕 𝑓 𝑥𝜕𝑥

𝑓 𝑥

The Canny edge detector 1) Smooth the input image with a Gaussian filter to remove

noise.

2) Compute the gradient magnitude and angle images.

3) Apply non-maxima suppression to the grad. magn. image. 4) Use double thresholding and connectivity analysis to detect

and link edges, i.e. thresholding with hysteres. Canny suggest the ratio 1:2 or 1:3 between the thresholds.

(5) A few edges thicker than 1 pixel may remain. Remove them with morphological thinning.)

𝐺 𝑥, 𝑦 𝑒 ⁄ (10-35)

𝑓 𝑥, 𝑦 𝐺 𝑥, 𝑦 ∗ 𝑓 𝑥, 𝑦 (10-36)

𝑀 𝑥, 𝑦 𝑔 𝑔 ,  where 𝑔 𝜕𝑓𝜕𝑥 , 𝑔 𝜕𝑓

𝜕𝑦 (10-37)

𝛼 𝑥, 𝑦 tan 𝑔 𝑔⁄ (10-38)𝛼 𝑥, 𝑦 arg 𝑔 , 𝑔

p. 18

Non-maxima suppression Let d1,d2,d3 and

d4 denote the 4 basic edge directions.

In every 3x3-region: Find the

direction dkthat is closest to (x,y).

If the value of M(x,y) is less than one of its two neighbors along dk, suppress edge.

Fig. 10.24

p. 19 Non-maxima suppression,example on whiteboard

p. 20

17 18

19 20

Page 6: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

6

Threshold with hysteresis Weak line segments with supporting strong line segments

are maintained. Weak line segments with no supporting strong line

segments are suppressed.

p. 21

Comparison of3 edge detection methods

orig

inal

Fig. 10.25

Mar

r-Hild

reth

Thre

shol

ded

mag

ni-

tude

of g

radi

ent o

f sm

ooth

ed im

age

Can

nyT L

=0.0

4, T

H=0

.10

filte

r siz

e:25

x25,

=4

p. 22

Comparison of3 edge detection methods

orig

inal

Fig. 10.26

Mar

r-Hild

reth

Thre

shol

ded

mag

ni-

tude

of g

radi

ent o

f 5x

5-sm

ooth

ed im

age

Can

nyT L

=0.0

5, T

H=0

.15

filte

r siz

e:13

x13,

=2

p. 23

Edge linking using gradient information (simple variant in one direction)

1) Compute the gradient magnitude M(x,y) and angle images (x,y) of the input image f(x,y).

2) Thesholding: g(x,y)=1 if M(x,y)>TM AND (x,y)=A±TA, otherwise g(x,y)=0.

3) Scan the rows of g and fill gaps smaller than a specified length.

4) Other directions can be processed similarly by rotating the image before step 2,3 and then rotating it back.

p. 24

21 22

23 24

Page 7: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

7

Edge linking using gradient information (simple case in the horizontal and vertical direction)

original

Fig. 10.27

M(x,y)h(x,y) horiz.

linking

v(x,y) vert. linking v(x,y) OR h(x,y) After thinning

p. 25 Edge linking using polygonial fit 1) Suppose that A

and B are end points and that all points are ordered.

2) A-B forms a line. The distance from this line to C is maximal and >T => C is a new vertex.

3) Process line A-C and line C-B as in step 2.

4) Continue in a similar way until finished.

Polygonial approximations can capture the essential shape with a simple description!

3rd ed:Fig. 10.28

p. 26

Edge linking using polygonial approximation. Applied on tooth.

Original, 550x566

3rd ed:Fig. 10.30

Gradient magnitude After morph. proc.

Afte

r pol

ygon

al

fit, T

=12.

Afte

r 1D

1x3

1 sm

ooth

ing

alon

g th

e cu

rve

a b g

j

Skel

eton

s ca

n be

ord

ered

with

ch

ain

code

!

k

p. 27 Global processing with the Hough transform𝑥 cos 𝜃 𝑦 sin 𝜃 𝜌

Fig. 10.29

(10-44)

Calculation of the Hough transform and its “inverse”

Lab 7

p. 28

25 26

27 28

Page 8: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

8

Hough transform calculation im-provements

Fig. 10.29

p. 29

Calculation of the gradient in the actual (green)point (xi,yi) gives . Insertion in (10-44) gives . Only the point () (blue) and a small neighbor-hood (blue dashed circle) in the Hough trans-form needs to be updated.

(10-44)𝑥 cos 𝜃 𝑦 sin 𝜃 𝜌

Faster. Less disturbing sinusoids in theHough transform.

Hough transform and inverse Hough transform With the Hough transform with

a point (x,y) in the image becomes a sinusoid in the Hough parameter space.

With the inverse Hough transform with

a point () in the Hough parameter space becomes a line in the image.

𝑥 cos 𝜃 𝑦 sin 𝜃 𝜌

𝑥 cos 𝜃 𝑦 sin 𝜃 𝜌

p. 30

Line detection withthe Hough transform (a) Take an image

with a weak non-connected line (here only 5 points) and possibly perform thresholding.

Hough transform gives (b)

Thresholding gives (c)

Inverse Hough transform gives (d)

𝑎

𝑐

𝑏

𝑑

p. 31 Line detection with the Hough transform a) Infrared image

of an airport. b) Thresholded

magnitude of the gradient.

c) Hough transform

d) Inverse Hough transform of the 3 strongest peeks in c) combined with linking of gaps< 5 pixels.Then overlaid on a).

From G&W 2:nd edition,Fig.10.21.

𝑎

𝑐

𝑏

𝑑

p. 32

29 30

31 32

Page 9: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

9

Line detection with the Hough transform a) Aerial airport image b) Edge image from

Canny’s algorithm c) Hough transform d) Inverse Hough

transform of peaks with correct angle.

e) Linking of gapsand overlaid on a). 𝑎

𝑐

𝑏

𝑑

Fig. 10.31

𝑒

p. 33 Hough transform and inverse Hough transform, alternative 2 With the Hough transform with

a point (x,y) in the image becomes a line in the Hough parameter space.

With the inverse Hough transform witha point (a,b) in the Hough parameter space becomes a line in the image.

𝑏 𝑥 ⋅ 𝑎 𝑦

𝑦 𝑎 ⋅ 𝑥 𝑏

For all (x,y) where f(x,y)!=0 For all (a,b) where b=-xa+y

H(a,b):= H(a,b) + f(x,y)END

END

Hough transform

p. 34

Calculation of the Hough trans-form and its inverse, alternative 2

Fig. 10.28

𝑏 𝑥 ⋅ 𝑎 𝑦

Problem: a and b ->

for some lines.

p. 35

𝑐 𝑦 ⋅ 𝑚 𝑥, 1 𝑚 1

Solution: double Hough spaces! Calcu-lation of the Hough transform, alt. 3𝑐 𝑥 ⋅ 𝑚 𝑦, 1 𝑚 1

p. 36

33 34

35 36

Page 10: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

10

𝑥 𝑦 ⋅ 𝑚 𝑐

Calculation of the inverse Hough transform, alternative 3

𝑦 𝑥 ⋅ 𝑚 𝑐

p. 37 Hough transform and inverse Hough transform of circles With the Hough transform

with a point () in the image becomes a cone ofcircles in the Hough parameter space.

With the inverse Hough transform with

a point (x,y) in the Hough parameter space becomes a cone of circles in the image.

For all () where b()!=0 For all (x,y,r) where (-x)2+(-y)2=r2

c(x,y,r):= c(x,y,r) + b()END

END

Hough transform

𝛼 𝑥 𝛽 𝑦 𝑟

𝛼 𝑥 𝛽 𝑦 𝑟 (10.2-39)

p. 38

Calculation of the Hough transform for circles

3D out-data space!

𝛼 𝑥 𝛽 𝑦 𝑟High

computational complexity!

p. 39

Fourier descriptors Let the boundary of an object

be represented as thesequence of coordinates

The DFT of s(k) is

The complex coefficients a(u) are called the Fourier descriptors of the boundary.

𝑠 𝑘 𝑥 𝑘 𝑗𝑦 𝑘

p. 40

(11-7)Fig. 11.18

𝑎 𝑢 𝑠 𝑘  𝑒 / , 𝑢 0,1,2, … , 𝐾 1 (11-8)

𝑠 𝑘 𝑥 𝑘 , 𝑦 𝑘 , 𝑘 0,1,2, . . . , 𝐾 1

37 38

39 40

Page 11: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

11

Fourier descriptors The inverse DFT of a(u) restores s(k):

If only the first P coefficients are used, the result is an approximation to s(k), namely

See Fredrik Larsson’s example (PhD from CVL) on next slide.

p. 41

(11-9)

(11-10)

𝑠 𝑘1𝐾 𝑎 𝑢  𝑒 /

𝑠 𝑘1𝑃 𝑎 𝑢  𝑒 /

Keeping different number of Fourier coefficients

p. 42

𝑃 2  ⇒

Properties of Fourier descriptors

Tab. 11.1

Fourier descriptor Invariances Translation affects the DC-component (zero frequency

component) only => remove DC-component Scaling affects the magnitude => normalize with the

(remaining) signal energy Rotation and index-shift affects the phase of each

coefficient => keep the phase and correlate, se next slide

p. 43 Fourier descriptors:Correlation of contour

The corre-lation max is the matching measure.

max(abs( )) gives rotation independent matching

max(real()) gives rotation dependent matching

𝑐 𝑐 𝑥 𝑐 𝛼 ⋅ 𝑐 𝑥 𝛼

𝑐 𝑐 𝑥 𝐷𝐹𝑇 𝐶 𝑢 ⋅ 𝐶 𝑢Variant 2

𝑐 𝑐

Which variant do

you prefer?

p. 44

start start

Variant 1

where 𝐶 𝑢 𝐷𝐹𝑇 𝑐 𝑥

and 𝐶 𝑢 𝐷𝐹𝑇 𝑐 𝑥

41 42

43 44

Page 12: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

12

Application of Fourier descriptors:Traffic sign Recognition see Fredrik Larsson’s PPT-

presentation “FourierDescr” at home page

Fredrik also mention that the traditional way to use Fourier descriptors is to remove the phase. This gives worse result.

p. 45

Eigenvectors The center of gravity

(mean) of the object mx=(m1,m2)is marked in red.

The main axes of the object oriented along the eigenvectorse1=(v11,v21)T and e2=(v12,v22)T are marked in green.

x1

x2

p. 46

Find translation and rotation by using eigenvectors, equations

𝐦𝐱1𝐾 𝐱𝐤 Compare (11-47)

From K vector samples, the mean vector can be approximated with:

𝐂𝐱1𝐾 𝐱𝐤 𝐦𝐱 𝐱𝐤 𝐦𝐱

1𝐾 𝐱𝐤𝐱𝐤

𝐓 𝐦𝐱𝐦𝐱𝐓

Compare (11-48)

Similarly, the covariance matrix can be approximated with:

p. 47 Simple equationsfor the previous image

𝑓𝑠𝑢𝑚 𝑓 𝑥 , 𝑥

𝑚∑ ∑ 𝑥 ⋅ 𝑓 𝑥 , 𝑥

𝑓𝑠𝑢𝑚

𝑚∑ ∑ 𝑥 ⋅ 𝑓 𝑥 , 𝑥

𝑓𝑠𝑢𝑚

var 𝑥∑ ∑ 𝑥 𝑚 ⋅ 𝑓 𝑥 , 𝑥

𝑓𝑠𝑢𝑚

var 𝑥∑ ∑ 𝑥 𝑚 ⋅ 𝑓 𝑥 , 𝑥

𝑓𝑠𝑢𝑚

cov 𝑥 , 𝑥 cov 𝑥 , 𝑥∑ ∑ 𝑥 𝑚 ⋅ 𝑥 𝑚 ⋅ 𝑓 𝑥 , 𝑥

𝑓𝑠𝑢𝑚

mx=(m1,m2) is the center of gravity (mean) of the object.

Set image f(x1,x2) to 1 inside object and to 0 outside object

p. 48

45 46

47 48

Page 13: Digital Image Processing p. 1 Edge models. p. 2 Edges ... · 1 Digital Image Processing Lecture 9 Edge detection The Marr-Hildrethedge detector Canny’sedge detector Edge linking

13

Find translation and rotation by using eigenvectors, equations

𝐂𝐱var 𝑥 cov 𝑥 , 𝑥

cov 𝑥 , 𝑥 var 𝑥 𝐀𝐓𝐂 𝐀 𝐕𝐃𝐕𝐓

 𝑣 𝑣𝑣 𝑣 ⋅ 𝑑 0

0 𝑑 ⋅𝑣 𝑣𝑣 𝑣

Matlab code:[V,D]=eigs(Cx)

𝑣𝑣  och 

𝑣𝑣   … are eigenvectors to Cx.

They gives the main axes of the object.

(11-51)𝐀𝐓 𝐀 𝟏

p. 49

Find translation and rotation by using eigenvectors, MATLAB-code

fsum = sum(sum(f)); m1 = sum(sum(x1Mat.*f))/fsum; m2 = sum(sum(x2Mat.*f))/fsum; Ixx = sum(sum((x1Mat-m1).^2.*f))/fsum; Iyy = sum(sum((x2Mat-m2).^2.*f))/fsum; Ixy = sum(sum((x1Mat-m1).*(x2Mat-m2).*f))/fsum; Iyx = sum(sum((x2Mat-m2).*(x1Mat-m1).*f))/fsum;

M = [Ixx Ixy; Iyx Iyy] [V D]=eigs(M)

The in-image is in f, x1Mat is a matrix with x1-coordinates, x2Mat is a matrix with x2-coordinates.

The main axis is in V!

p. 50

Find rotation by using eigenvectors, G&W example

Fig. 11.43

The points in the region (or its boundary) may be treated as two-dimensional vectors, x=(x1,x2)T, where x1 and x2 are the coordinate values along the x1-

and x2-axis, respectively.

p. 51

49 50

51