2d transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2d transformations. • projective...
TRANSCRIPT
![Page 1: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/1.jpg)
2D transformations (a.k.a. warping)
16-385 Computer VisionSpring 2020, Lecture 7http://www.cs.cmu.edu/~16385/
![Page 2: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/2.jpg)
Course announcements• Homework 1 posted on course website.
- Due on February 5th at 23:59.- This homework is in Matlab.- How many of you have looked at/started/finished homework 1?
• Homework 2 will be posted tonight on the course website.- It will be due on February 19th at 23:59 pm.- Start early because it is much larger and more difficult than homework 1.
• Second theory quiz on course website and due on February 10th, at 23:59.
• Starting with quiz 3, release/due dates will be on Sunday.
• Reminder: All office hours are at the Smith Hall 200 conference room (not my office!).
• How was the first quiz?
![Page 3: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/3.jpg)
Overview of today’s lecture
• Reminder: image transformations.
• 2D transformations.
• Projective geometry 101.
• Transformations in projective geometry.
• Classification of 2D transformations.
• Determining unknown 2D transformations.
• Determining unknown image warps.
![Page 4: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/4.jpg)
Slide credits
Most of these slides were adapted from:
• Kris Kitani (16-385, Spring 2017).
![Page 5: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/5.jpg)
Reminder: image transformations
![Page 6: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/6.jpg)
What is an image?
A (grayscale) image is a 2D
function.
domainWhat is the range of the image function f?
grayscale image
![Page 7: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/7.jpg)
What types of image transformations can we do?
changes pixel values changes pixel locations
Filtering Warping
![Page 8: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/8.jpg)
What types of image transformations can we do?
changes range of image function changes domain of image function
Filtering Warping
![Page 9: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/9.jpg)
Warping example: feature matching
![Page 10: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/10.jpg)
Warping example: feature matching
![Page 11: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/11.jpg)
Warping example: feature matching
How do you compute the transformation?
• object recognition• 3D reconstruction• augmented reality• image stitching
![Page 12: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/12.jpg)
Warping example: feature matching
Given a set of matched feature points:
and a transformation:
find the best estimate of the parameters
parameterstransformation
function
point in one image
point in the other image
What kind of transformation functions are there?
![Page 13: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/13.jpg)
2D transformations
![Page 14: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/14.jpg)
2D transformations
translation rotation aspect
affine perspective cylindrical
![Page 15: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/15.jpg)
2D planar transformations
![Page 16: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/16.jpg)
• Each component multiplied by a scalar• Uniform scaling - same scalar for each component
Scale
How would you implement scaling?
![Page 17: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/17.jpg)
• Each component multiplied by a scalar• Uniform scaling - same scalar for each component
ScaleWhat’s the effect of using
different scale factors?
![Page 18: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/18.jpg)
• Each component multiplied by a scalar• Uniform scaling - same scalar for each component
Scale
scaling matrix S
matrix representation of scaling:
![Page 19: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/19.jpg)
Shear
How would you implement shearing?
![Page 20: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/20.jpg)
Shear or in matrix form:
![Page 21: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/21.jpg)
rotation around the origin
How would you implement rotation?
![Page 22: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/22.jpg)
rotation around the origin
![Page 23: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/23.jpg)
rotation around the origin
Polar coordinates…x = r cos (φ)y = r sin (φ)x’ = r cos (φ + θ)y’ = r sin (φ + θ)
Trigonometric Identity…x’ = r cos(φ) cos(θ) – r sin(φ) sin(θ)y’ = r sin(φ) cos(θ) + r cos(φ) sin(θ)
Substitute…x’ = x cos(θ) - y sin(θ)y’ = x sin(θ) + y cos(θ)
φ
𝑟
𝑟
![Page 24: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/24.jpg)
rotation around the origin
or in matrix form:
![Page 25: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/25.jpg)
2D planar and linear transformations
point
𝒙′ = 𝑓 𝒙; 𝑝
𝑥′
𝑦′= 𝑴
𝑥𝑦
parameters 𝑝 𝒙
![Page 26: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/26.jpg)
2D planar and linear transformations
Scale
Rotate
Shear
Flip across y
Flip across origin
Identity
![Page 27: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/27.jpg)
2D translation
How would you implement translation?
![Page 28: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/28.jpg)
2D translation
What about matrix representation?
![Page 29: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/29.jpg)
2D translation
What about matrix representation?
Not possible.
![Page 30: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/30.jpg)
Projective geometry 101
![Page 31: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/31.jpg)
Homogeneous coordinates
• Represent 2D point with a 3D vector
add a 1 here𝑥𝑦 ֜
𝑥𝑦1
heterogeneous coordinates
homogeneous coordinates
![Page 32: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/32.jpg)
𝑥𝑦 ֜
𝑥𝑦1
≝𝑎𝑥𝑎𝑦𝑎
Homogeneous coordinates
• Represent 2D point with a 3D vector• 3D vectors are only defined up to scale
heterogeneous coordinates
homogeneous coordinates
![Page 33: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/33.jpg)
2D translation
What about matrix representation using homogeneous coordinates?
![Page 34: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/34.jpg)
2D translation
What about matrix representation using heterogeneous coordinates?
![Page 35: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/35.jpg)
2D translation using homogeneous coordinates
![Page 36: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/36.jpg)
Homogeneous coordinatesConversion:
• heterogeneous → homogeneous
• homogeneous → heterogeneous
• scale invariance
Special points:
• point at infinity
• undefined
![Page 37: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/37.jpg)
Projective geometry
image plane
X is a projection of a point P on the image plane
image point in pixel coordinates
image point in homogeneous
coordinates
What does scaling X correspond to?
![Page 38: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/38.jpg)
Transformations in projective geometry
![Page 39: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/39.jpg)
2D transformations in heterogeneous coordinates
Re-write these transformations as 3x3 matrices:
translation
rotation shearing
scaling
?
??
![Page 40: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/40.jpg)
2D transformations in heterogeneous coordinates
Re-write these transformations as 3x3 matrices:
translation
rotation shearing
scaling
??
![Page 41: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/41.jpg)
2D transformations in heterogeneous coordinates
Re-write these transformations as 3x3 matrices:
translation
rotation shearing
scaling
?
![Page 42: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/42.jpg)
2D transformations in heterogeneous coordinates
Re-write these transformations as 3x3 matrices:
translation
rotation shearing
scaling
![Page 43: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/43.jpg)
Matrix composition
Transformations can be combined by matrix multiplication:
p’ = ? ? ? p
![Page 44: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/44.jpg)
Matrix composition
Transformations can be combined by matrix multiplication:
p’ = translation(tx,ty) rotation(θ) scale(s,s) p
Does the multiplication order matter?
![Page 45: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/45.jpg)
Classification of 2D transformations
![Page 46: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/46.jpg)
Classification of 2D transformations
![Page 47: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/47.jpg)
Classification of 2D transformations
?
?
?
?
?
![Page 48: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/48.jpg)
Classification of 2D transformations
Translation:
How many degrees of freedom?
![Page 49: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/49.jpg)
Classification of 2D transformations
Euclidean (rigid):rotation + translation
Are there any values that are related?
![Page 50: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/50.jpg)
Classification of 2D transformations
Euclidean (rigid):rotation + translation
How many degrees of freedom?
![Page 51: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/51.jpg)
Classification of 2D transformations
Euclidean (rigid):rotation + translation
which other matrix values will change if this increases?
![Page 52: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/52.jpg)
Classification of 2D transformations
Euclidean (rigid):rotation + translation
what will happen to the image if this increases?
![Page 53: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/53.jpg)
Classification of 2D transformations
Euclidean (rigid):rotation + translation
what will happen to the image if this increases?
![Page 54: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/54.jpg)
Classification of 2D transformations
Similarity:uniform scaling + rotation
+ translation
Are there any values that are related?
![Page 55: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/55.jpg)
Classification of 2D transformations
multiply these four by scale s
How many degrees of freedom?
Similarity:uniform scaling + rotation
+ translation
![Page 56: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/56.jpg)
Classification of 2D transformations
what will happen to the image if this increases?
Similarity:uniform scaling + rotation
+ translation
![Page 57: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/57.jpg)
Classification of 2D transformations
Affine transform:uniform scaling + shearing
+ rotation + translation
Are there any values that are related?
![Page 58: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/58.jpg)
Classification of 2D transformations
Affine transform:uniform scaling + shearing
+ rotation + translation
Are there any values that are related?
similarity shear
![Page 59: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/59.jpg)
Classification of 2D transformations
Affine transform:uniform scaling + shearing
+ rotation + translation
How many degrees of freedom?
similarity shear
![Page 60: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/60.jpg)
Affine transformations
Affine transformations are combinations of
• arbitrary (4-DOF) linear transformations; and
• translations
Properties of affine transformations:
• origin does not necessarily map to origin
• lines map to lines
• parallel lines map to parallel lines
• ratios are preserved
• compositions of affine transforms are also affine transforms
Does the last coordinate w ever change?
![Page 61: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/61.jpg)
Affine transformations
Affine transformations are combinations of
• arbitrary (4-DOF) linear transformations; and
• translations
Properties of affine transformations:
• origin does not necessarily map to origin
• lines map to lines
• parallel lines map to parallel lines
• ratios are preserved
• compositions of affine transforms are also affine transforms
Nope! But what does that mean?
![Page 62: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/62.jpg)
How to interpret affine transformations here?
image plane
X is a projection of a point P on the image plane
image point in pixel coordinates
image point in heterogeneous
coordinates
![Page 63: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/63.jpg)
Projective transformations
Projective transformations are combinations of
• affine transformations; and
• projective wraps
Properties of projective transformations:
• origin does not necessarily map to origin
• lines map to lines
• parallel lines do not necessarily map to parallel lines
• ratios are not necessarily preserved
• compositions of projective transforms are also projective
transforms
How many degrees of freedom?
![Page 64: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/64.jpg)
Projective transformations
Projective transformations are combinations of
• affine transformations; and
• projective wraps
Properties of projective transformations:
• origin does not necessarily map to origin
• lines map to lines
• parallel lines do not necessarily map to parallel lines
• ratios are not necessarily preserved
• compositions of projective transforms are also projective
transforms
8 DOF: vectors (and therefore matrices) are defined up to scale)
![Page 65: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/65.jpg)
How to interpret projective transformations here?
image plane
X is a projection of a point P on the image plane
image point in pixel coordinates
image point in heterogeneous
coordinates
![Page 66: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/66.jpg)
Determining unknown 2D transformations
![Page 67: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/67.jpg)
Determining unknown transformations
A
D
B E
F
C
Suppose we have two triangles: ABC and DEF.
![Page 68: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/68.jpg)
Determining unknown transformations
A
D
B E
F
C
Suppose we have two triangles: ABC and DEF.• What type of transformation will map A to D, B to E, and C to F?
![Page 69: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/69.jpg)
Determining unknown transformations
A
D
B E
F
C
Suppose we have two triangles: ABC and DEF.• What type of transformation will map A to D, B to E, and C to F?• How do we determine the unknown parameters?
Affine transform:uniform scaling + shearing
+ rotation + translation
How many degrees of freedom do we have?
![Page 70: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/70.jpg)
Determining unknown transformations
A
D
B E
F
C
Suppose we have two triangles: ABC and DEF.• What type of transformation will map A to D, B to E, and C to F?• How do we determine the unknown parameters?
• One point correspondence gives how many equations?
• How many point correspondences do we need?point correspondences
unknowns
![Page 71: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/71.jpg)
Determining unknown transformations
A
D
B E
F
C
Suppose we have two triangles: ABC and DEF.• What type of transformation will map A to D, B to E, and C to F?• How do we determine the unknown parameters?
How do we solve this for M?
point correspondences
unknowns
![Page 72: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/72.jpg)
Least Squares Error
![Page 73: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/73.jpg)
What is
this?
Least Squares Error
What is
this?
What is
this?
![Page 74: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/74.jpg)
predictedlocation
Least Squares Error
measured location
Euclidean (L2) normEuclidean (L2) norm
squared!
![Page 75: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/75.jpg)
Residual (projection error)
Least Squares Error
![Page 76: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/76.jpg)
What is the free variable?
What do we want to optimize?
Least Squares Error
![Page 77: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/77.jpg)
Find parameters that minimize squared error
![Page 78: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/78.jpg)
General form of linear least squares
(matrix form)
(Warning: change of notation. x is a vector of parameters!)
![Page 79: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/79.jpg)
Determining unknown transformations
Affine transformation:
Vectorize transformation parameters:
Notation in system form:
Stack equations from point correspondences:
Why can we drop the last line?
![Page 80: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/80.jpg)
General form of linear least squares
(matrix form)
(Warning: change of notation. x is a vector of parameters!)
This function is quadratic.How do you find the root of a quadratic?
![Page 81: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/81.jpg)
Solving the linear system
Convert the system to a linear least-squares problem:
Expand the error:
Set derivative to 0
Minimize the error:
Solve for x
In Matlab:
x = A \ b
Note: You almost never want to compute the inverse of a matrix.
![Page 82: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/82.jpg)
Linear least squares estimation only works when the transform function is ?
![Page 83: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/83.jpg)
Linear least squares estimation only works when the transform function is linear! (duh)
Also doesn’t deal well with outliers
![Page 84: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/84.jpg)
Determining unknown image warps
![Page 85: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/85.jpg)
Determining unknown image warps
Suppose we have two images.• How do we compute the transform that takes one to the other?
x x’
T(x, y)
f(x, y) g(x’, y’)
y y’
![Page 86: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/86.jpg)
Forward warping
Suppose we have two images.• How do we compute the transform that takes one to the other?
x x’
T(x, y)
f(x, y) g(x’, y’)
y y’
1. Form enough pixel-to-pixel correspondences between two images
2. Solve for linear transform parameters as before
3. Send intensities f(x,y) in first image to their corresponding location in the second image
later lecture
![Page 87: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/87.jpg)
Forward warping
Suppose we have two images.• How do we compute the transform that takes one to the other?
x x’
T(x, y)
f(x, y) g(x’, y’)
y y’
1. Form enough pixel-to-pixel correspondences between two images
2. Solve for linear transform parameters as before
3. Send intensities f(x,y) in first image to their corresponding location in the second image
what is the problem with this?
![Page 88: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/88.jpg)
Forward warping
Pixels may end up between two points• How do we determine the intensity of each point?
f(x,y) g(x’,y’)
T(x,y)
x x’
y y’
![Page 89: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/89.jpg)
Forward warping
Pixels may end up between two points• How do we determine the intensity of each point?
f(x,y) g(x’,y’)
T(x,y)
x x’
y y’
✓ We distribute color among neighboring pixels (x’,y’) (“splatting”)
• What if a pixel (x’,y’) receives intensity from more than one pixels (x,y)?
![Page 90: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/90.jpg)
Forward warping
Pixels may end up between two points• How do we determine the intensity of each point?
f(x,y) g(x’,y’)
T(x,y)
x x’
y y’
✓ We distribute color among neighboring pixels (x’,y’) (“splatting”)
• What if a pixel (x’,y’) receives intensity from more than one pixels (x,y)?
✓ We average their intensity contributions.
![Page 91: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/91.jpg)
Inverse warping
Suppose we have two images.• How do we compute the transform that takes one to the other?
x x’
T-1(x, y)
f(x, y) g(x’, y’)
y y’
1. Form enough pixel-to-pixel correspondences between two images
2. Solve for linear transform parameters as before, then compute its inverse
3. Get intensities g(x',y') in in the second image from point (x,y) = T-1(x’,y’) in first image
what is the problem with this?
![Page 92: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/92.jpg)
Inverse warping
Pixel may come from between two points• How do we determine its intensity?
f(x,y) g(x’,y’)
T-1(x,y)
x x’
y y’
![Page 93: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/93.jpg)
Inverse warping
Pixel may come from between two points• How do we determine its intensity?
f(x,y) g(x’,y’)
T-1(x,y)
x x’
y y’
✓ Use interpolation
![Page 94: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/94.jpg)
Bilinear interpolation
1.Interpolate to find R2
2.Interpolate to find R1
3.Interpolate to find P
Grayscale example
In matrix form (with adjusted coordinates)
In Matlab:
call interp2
![Page 95: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/95.jpg)
Forward vs inverse warping
Suppose we have two images.• How do we compute the transform that takes one to the other?
x x’
T(x, y)
f(x, y) g(x’, y’)
y y’
T-1(x, y)
Pros and cons of each?
![Page 96: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/96.jpg)
Forward vs inverse warping
Suppose we have two images.• How do we compute the transform that takes one to the other?
x x’
T(x, y)
f(x, y) g(x’, y’)
y y’
T-1(x, y)
• Inverse warping eliminates holes in target image• Forward warping does not require existence of inverse transform
![Page 97: 2D transformations (a.k.a. warping)16385/lectures/lecture7.pdf• 2D transformations. • Projective geometry 101. • Transformations in projective geometry. • Classification of](https://reader033.vdocuments.net/reader033/viewer/2022042808/5f867b048135f930cd2d064d/html5/thumbnails/97.jpg)
References
Basic reading:• Szeliski textbook, Section 3.6.
Additional reading:• Hartley and Zisserman, “Multiple View Geometry in Computer Vision,” Cambridge University Press 2004.
a comprehensive treatment of all aspects of projective geometry relating to computer vision, and also a very useful reference for the second part of the class.
• Richter-Gebert, “Perspectives on projective geometry,” Springer 2011.a beautiful, thorough, and very accessible mathematics textbook on projective geometry (available online for free from CMU’s library).