cs 691b computational photography instructor: gianfranco doretto image warping
TRANSCRIPT
![Page 1: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/1.jpg)
CS 691B Computational Photography
Instructor: Gianfranco DorettoImage Warping
![Page 2: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/2.jpg)
Image Transformations
Image Filtering: change range of imageg(x) = T(f(x))
f
x
Tf
x
f
x
Tf
x
Image Warping: change domain of imageg(x) = f(T(x))
![Page 3: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/3.jpg)
Image Transformations
T
T
f
f g
g
Image Filtering: change range of imageg(x) = T(f(x))
Image Warping: change domain of imageg(x) = f(T(x))
![Page 4: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/4.jpg)
Parametric (global) warping
Examples of parametric warps:
translation rotation aspect
affineperspective
cylindrical
![Page 5: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/5.jpg)
Parametric (global) warping
Transformation T is a coordinate-changing machine:p’ = T(p)
What does it mean that T is global?Is the same for any point pcan be described by just a few numbers (parameters)
Let’s represent T as a matrix: p’ = Mp
T
p = (x,y) p’ = (x’,y’)
![Page 6: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/6.jpg)
Scaling• Scaling a coordinate means multiplying each of its
components by a scalar• Uniform scaling means this scalar is the same for all
components:
x 2
![Page 7: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/7.jpg)
• Non-uniform scaling: different scalars per component:•
Scaling
X x 2,Y x 0.5
![Page 8: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/8.jpg)
Scaling
Scaling operation:
Or, in matrix form:
byy
axx
'
'
y
x
b
a
y
x
0
0
'
'
scaling matrix S
What’s inverse of S?
![Page 9: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/9.jpg)
2-D Rotation
θ
(x, y)
(x’, y’)
x’ = x cos(θ) - y sin(θ)y’ = x sin(θ) + y cos(θ)
![Page 10: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/10.jpg)
2-D Rotation
x = r cos (ϕ)y = r sin (ϕ)x’ = r cos (ϕ + θ)y’ = r sin (ϕ + θ)
Trig Identity…x’ = r cos(ϕ) cos(θ) – r sin(ϕ) sin(θ)y’ = r sin(ϕ) cos(θ) + r cos(ϕ) sin(θ)
Substitute…x’ = x cos(θ) - y sin(θ)y’ = x sin(θ) + y cos(θ)θ
(x, y)
(x’, y’)
ϕ
![Page 11: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/11.jpg)
2-D Rotation•This is easy to capture in matrix form:
•Even though sin(θ) and cos(θ) are nonlinear functions of θ,– x’ is a linear combination of x and y– y’ is a linear combination of x and y
•What is the inverse transformation?– Rotation by –θ– For rotation matrices
y
x
y
x
cossin
sincos
'
'
TRR 1
R
![Page 12: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/12.jpg)
2x2 Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Identity?
yyxx
''
yx
yx
1001
''
2D Scale around (0,0)?
ysy
xsx
y
x
*'
*'
y
x
s
s
y
x
y
x
0
0
'
'
![Page 13: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/13.jpg)
2x2 Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Rotate around (0,0)?
yxyyxx
*cos*sin'*sin*cos'
y
x
y
x
cossin
sincos
'
'
2D Shear?
yxshy
yshxx
y
x
*'
*'
y
x
sh
sh
y
x
y
x
1
1
'
'
![Page 14: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/14.jpg)
2x2 Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Mirror about Y axis?
yyxx
''
yx
yx
1001
''
2D Mirror over (0,0)?
yyxx
''
yx
yx
1001
''
![Page 15: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/15.jpg)
2x2 Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Translation?
y
x
tyy
txx
'
'
Only linear 2D transformations can be represented with a 2x2 matrix
NO!
![Page 16: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/16.jpg)
All 2D Linear Transformations
• Linear transformations are combinations of …– Scale,– Rotation,– Shear, and– Mirror
• Properties of linear transformations:– Origin maps to origin– Lines map to lines– Parallel lines remain parallel– Ratios are preserved– Closed under composition
y
x
dc
ba
y
x
'
'
yx
lkji
hgfe
dcba
yx
''
![Page 17: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/17.jpg)
Consider a different Basisv =(vi,vj)
u=(ui,uj)
puv
j =(0,1)
i =(1,0)
pij
pij=5i+4j = (5,4) puv = 4u+3v = (4,3)
![Page 18: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/18.jpg)
Linear Transformations as Change of Basis
Any linear transformation is a change of basis!!!
j =(0,1)
i =(1,0)
pij = 4u+3v = (pi,pj) ?
pi = 4ui+3vi
pj = 4uj+3vj
v =(vi,vj)
u=(ui,uj)
puv pij
puv = (4,3)
puvpij ==jj
ii
jj
ii
vuvu
vuvu
34
![Page 19: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/19.jpg)
What’s the inverse transform?
• How can we change from any basis to any basis?• What if the basis are orthonormal? And orthogonal?
v =(vi,vj)
u=(ui,uj)
puv
j =(0,1)
i =(1,0)
pij = (5,4)
pij
puv = (pu,pv) = ?= puu + pvv
pijpuv ==jj
ii
jj
ii
vuvu
vuvu
45
-1-1
![Page 20: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/20.jpg)
Projection onto orthonormal basisv =(vi,vj)
u=(ui,uj)
puv
j =(0,1)
i =(1,0)
pij = (5,4)
pij
puv = (u·pij, v·pij)
pijpuv ==ji
ji
ji
ji
vvuu
vvuu
45
NOTE: The inverse is just the transpose!
![Page 21: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/21.jpg)
Homogeneous Coordinates
• Q: How can we represent translation as a 3x3 matrix?
y
x
tyy
txx
'
'
![Page 22: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/22.jpg)
Homogeneous Coordinates
Represent coordinates in 2 dimensions with a 3-vector
1
coords shomogeneou y
x
y
x
![Page 23: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/23.jpg)
Homogeneous Coordinates• Add a 3rd coordinate to every 2D point
– (x, y, w) represents a point at location (x/w, y/w)
– (x, y, 0) represents a point at infinity or a vector
– (0, 0, 0) is not allowed
Convenient coordinate system to represent many useful transformations
1 2
1
2(2,1,1) or (4,2,2) or (6,3,3)
x
y
![Page 24: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/24.jpg)
Homogeneous Coordinates
• Q: How can we represent translation as a 3x3 matrix?
• A: Using the rightmost column:
100
10
01
y
x
t
t
ranslationT
y
x
tyy
txx
'
'
![Page 25: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/25.jpg)
Translation
Example of translation
11100
10
01
1
'
'
y
x
y
x
ty
tx
y
x
t
t
y
x
tx = 2ty = 1
Homogeneous Coordinates
![Page 26: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/26.jpg)
Basic 2D Transformations
Basic 2D transformations as 3x3 matrices
1100
0cossin
0sincos
1
'
'
y
x
y
x
1100
10
01
1
'
'
y
x
t
t
y
x
y
x
1100
01
01
1
'
'
y
x
sh
sh
y
x
y
x
Translate
Rotate Shear
1100
00
00
1
'
'
y
x
s
s
y
x
y
x
Scale
![Page 27: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/27.jpg)
Matrix Composition
Transformations can be combined by matrix multiplication
wyx
sysx
tytx
wyx
1000000
1000cossin0sincos
1001001
'''
p’ = T(tx,ty) R(Q) S(sx,sy) p
Does the order of multiplication matter?
![Page 28: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/28.jpg)
Affine Transformations
• Affine transformations are combinations of …– Linear transformations, and– Translations
• Properties of affine transformations:– Origin does not necessarily map to origin– Lines map to lines– Parallel lines remain parallel– Ratios are preserved– Closed under composition– Model change of basis
• Will the last coordinate w always be 1?
wyx
fedcba
wyx
100''
![Page 29: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/29.jpg)
Projective Transformations
• Projective transformations …– Affine transformations, and– Projective warps
• Properties of projective transformations:– Origin does not necessarily map to origin– Lines map to lines– Parallel lines do not necessarily remain parallel– Ratios are not preserved– Closed under composition– Models change of basis
wyx
ihgfedcba
wyx
'''
![Page 30: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/30.jpg)
2D image transformations
These transformations are a nested set of groups• Closed under composition and inverse is a member
![Page 31: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/31.jpg)
Recovering Transformations
• What if we know f and g and want to recover the transform T?– willing to let user provide
correspondences• How many do we need?
x x’
T(x,y)y y’
f(x,y) g(x’,y’)
?
![Page 32: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/32.jpg)
Translation: # correspondences?
• How many Degrees of Freedom?• How many correspondences needed for translation?• What is the transformation matrix?
x x’
T(x,y)y y’
?
100
'10
'01
yy
xx
pp
pp
M
![Page 33: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/33.jpg)
Euclidian: # correspondences?
• How many DOF?• How many correspondences needed for
translation+rotation?
x x’
T(x,y)y y’
?
![Page 34: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/34.jpg)
Affine: # correspondences?
• How many DOF?• How many correspondences needed
for affine?
x x’
T(x,y)y y’
?
![Page 35: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/35.jpg)
Projective: # correspondences?
• How many DOF?• How many correspondences needed for
projective?
x x’
T(x,y)y y’
?
![Page 36: CS 691B Computational Photography Instructor: Gianfranco Doretto Image Warping](https://reader036.vdocuments.net/reader036/viewer/2022070413/5697bfab1a28abf838c9ae7b/html5/thumbnails/36.jpg)
Slide Credits
• This set of sides also contains contributionskindly made available by the following authors– Alexei Efros– Richard Szelisky