announcements image matching!

2/3/16

1

Image Matching!

CS 510 Lecture #5

February 3rd, 2016

Announcements

• PA1 is due in 1 week– Leave time to learn OpenCV– Think of & implement something creative

2/3/16 2

So far, we have been covering background material…• Geometric Image Transformations• Fourier Analysis

• We haven’t done any vision yet!• OK, lets consider two images…• ... are they similar?

2/3/16 3

Compare a big image to a small image

• Issues:– Geometric

manipulation– Differences in

information content

– Pattern comparison

2/3/16 4

Image Transformation• I(x,y) = I’(G�[x,y]T)

• Simple for continuous, infinite images

• Problematic for discrete, finite images

Source & Destination Images• We apply a transformation to a source image

to produce a destination image• The role of source & destination are not

symmetric– We need to know where every destination pixel

came from in the source image• Typically, a non-integer location

– We do not need to know where every source pixel went

• It could be off the image, or between pixels, or….

2/3/16

2

Basic Transformation Algorithm

For every (x,y) in dest

{

real (u,v) = G-1 (x,y)

pixel p = Interpolate(Src, u, v)

dest(x,y) = p

}

Applying Transformations

• I assume you can invert a 3x3 matrix• So the trick is interpolation. 3 forms:

– Nearest Neighbor (fast, bad)– Bilinear (less fast, good)– Bicubic (slowest, best)

• OpenCV supports all three forms

Nearest Neighbor Interpolation

• (u’, v’) = G-1 (x, y, 1)• u = round(u’)• v = round(v’)• Interpolate(Src, x, y) = Src[u,v]

In terms of Fourier Analysis, this is awful in the frequency domain

Bilinear Interpolation(0,0) (1,0)

(0,1) (1,1)

X(u,v)

(u,0)

(u,1)

Linearly Interpolate along row

Linearly Interpolate along row

(u,v)

Bilinear Interpolation (II)• Bilinear interpolation is actually a product of two

linear interpolati ons– … and therefore non-linear

• Equivalent to a least-squar es approximation if you put a plane through the four neighbori ng points

• Typical expression:

• Linear algebraic expression

Bicubic Interpolation

• Product of two cubic interpolations– 1 in x, 1 in y

• Based on a 4x4 grid of neighboring pixels• In each dimension, create a cubic curve

that exactly interpolates all four points– Similar to Bezier curves in graphics– Except curve passes through all 4 points

2/3/16

3

Bicubic Interpolation (II)(-1,-1) (2,-1)

(-1,2) (2,2)

X(u,v)

Bicubic Interpolation (III)• The equation of a cubic function is:

• This can be rewritten as:

• We know the values of f at -1,0,1,2

Bicubic Interpolation (IV)• Therefore:

• And:

Bicubic Interpolation (V)

• To interpolate a value:1. Interpolate along the four rows2. Interpolate the results vertically

• Each interpolation is a matrix/vector multiply– 16 mults, 12 adds per interpolation– 80 mults, 60 adds overall

The Interpolation Anti-climax• You don’t need to implement geometric

transformations of interpolati ons• OpenCV supports geometric transformations

– warpAffine applies an affine transformation– warpPerspective applies a perspective transformation– Both give you the option of interpolation technique

• Nearest Neighbor• Bilinear• Bicubic

• The point of the lecture is to know what is happening when you use them

Back to this:

• Issues:– Geometric

manipulation– Differences in

information content

– Pattern comparison

2/3/16 18

2/3/16

4

Will the pixels match?• Given

– a pixel Is(x,y) in the small image– And the interpolated pixel Ib(G-1(x,y))

• Will they be the same?– Given NN interpolation?– Given bilinear interpolation?– Given bicubic interpolation?

• Why or why not?

2/3/16 19

Frequencies & Scale

2/3/16 20

𝐼" 𝑥,𝑦 =1𝑁) ) 𝐹" 𝑢,𝑣 𝑐𝑜𝑠

2𝜋𝑢𝑣𝑥𝑁

+ 𝑖𝑠𝑖𝑛2𝜋𝑢𝑥𝑣𝑁

5 67

89:5 67

5 67

;9<

+𝑁" 𝑥,𝑦

ContributionoftermsbelowNyquistrate

NoisecreatedbyaliasedenergyaboveNyquistrate

Frequencies & Scale (cont.)

2/3/16 21

𝐼= 𝐺:? 𝑥,𝑦 =

1𝑀) ) 𝐹= 𝑢,𝑣 𝑐𝑜𝑠

2𝜋𝑢𝑣𝑥𝑀

+ 𝑖𝑠𝑖𝑛2𝜋𝑢𝑥𝑣𝑀

A 67

89:A 67

A 67

;9<

+𝑁= 𝑥, 𝑦

NotethatM>N,somorefrequencies

Slightlylessaliasednoise

The difference is…

2/3/16 22

𝐼= 𝐺:? 𝑥, 𝑦 −𝐼" 𝑥, 𝑦 =

1𝑀 ) ) 𝐹= 𝑢,𝑣 𝑐𝑜𝑠

2𝜋𝑢𝑣𝑥𝑀 +𝑖𝑠𝑖𝑛

2𝜋𝑢𝑥𝑣𝑀

A 67

8C5 67

A 67

;C5 67

+

1𝑀 ) ) 𝐹= 𝑢, 𝑣 𝑐𝑜𝑠

2𝜋𝑢𝑣𝑥𝑀 + 𝑖𝑠𝑖𝑛

2𝜋𝑢𝑥𝑣𝑀

:5 67

8C:A 67

A 67

;C5 67

+

𝑁= 𝐺:? 𝑥, 𝑦 −𝑁" 𝑥,𝑦

FrequenciesbetweentheNyquistrates

So to compare across scales

• Remove frequencies from the higher-scaled image– Remove all frequencies above the lower

Nyquist rate

• But how?

2/3/16 23

Low-Pass Filtering Approach #1

2/3/16 CS510,ImageComputation,©Ross Beveridge&BruceDraper 24

• Drop high frequency Fourier coefficients.To low-pass filter an image:1) convert to frequency domain2) discard all values for u > thresh3) Convert back to spatial domain

Brainflux FourierApplethttp://www.brainflux.org/java/classes /FFT2DApplet.html

Ø Butthereisabetterway.

2/3/16

5

Convolution


“Slide” a mask over an image. At each window position, multiply the mask values by the image value under them.Sum the resultsfor every pixel.

Think of this as a sliding dot product

Wearrive atthe fundamentalideaofconvolution.

Convolution (II)


• Formally, convolution is often expressed as follows:

Ø Ofcourse,wearedealingwith finite,discretefunctions:

Convolution Examples• Let F1 = [1,2,3,4,5]• Let F2 = [1,2,1,2,1]• Let G1 = [-1,2,-1]• Let G2 = [1/3,1/3,1/3]

• Then F1*G1=[0,0,0,0,6] • Then F2*G1=[0,2,-2,2,0]• Then F1*G2=[1,2,3,4,3]• Then F2*G2=[1,4/3,5/3,4/3,1]


Convolution (III)

• Why introduce convolution now?

• Because multiplying two Fourier transforms in the frequency domain is the same as convolving their inverse Fourier transforms in the spatial domain! (trust me)


Convolution Theorem

2/3/16 29

“Inmathematics,theconvolutiontheoremstatesthatundersuitableconditionstheFouriertransformofaconvolutionisthepointwise productofFouriertransforms.”

announcements image matching!

Documents