image morphing
TRANSCRIPT
Image Morphing
Michael Kazhdan
(601.457/657)
HB Ch. 16.5
Feature Based Image Metamorphosis, Beier and Neely 1992
Image Morphingβ’ Animate transition between two images
H&B Figure 16.9
Image Morphingβ’ Animate transition between two images
H&B Figure 16.9
Two Components:
β’ Blending
β’ Warping
BlendingCombine images using an πΌπΌ-blend (πΌπΌ β 0,1 ):
blend ππ, ππ,πΌπΌ = 1 β πΌπΌ β Img1 ππ, ππ + πΌπΌ β Img2(ππ, ππ)
πΌπΌ = 0.0 πΌπΌ = 1.0πΌπΌ = 0.5
Img1 Img2blend
πΌπΌ
Image WarpingDeform Img1 so its shape matches that of Img2β¦
Img1
Img2
πΌπΌ = 0.0 πΌπΌ = 1.0πΌπΌ = 0.5 πΌπΌ
Image WarpingDeform Img1 so its shape matches that of Img2β¦
warpwarp
πΌπΌ = 0.0 πΌπΌ = 1.0πΌπΌ = 0.5 πΌπΌ
Img1
Img2
Image WarpingDeform Img2 so its shape matches that of Img1β¦
warp warp
warpwarp
πΌπΌ = 0.0 πΌπΌ = 1.0πΌπΌ = 0.5 πΌπΌ
Img1
Img2
Image Morphing⦠then blend
warp warp
warpwarp
blend
πΌπΌ = 0.0 πΌπΌ = 1.0πΌπΌ = 0.5 πΌπΌ
Img1
Img2
Image Morphingβ’ The warping step is the hard one
Aim to align features in images
H&B Figure 16.9
How do we specifythe mapping for the warp?
Feature-Based Warpingβ’ [Beier & Neeley, 1992] use a pair of lines to specify
the warp Given ππ in destination image, where is πππ in the source? Describe ππ relative to the destination line Map the description to the source
Source image Destination image
ππ
π’π’
π£π£
π’π’ is a signed fractionπ£π£ is a signed length (in pixels)
Beier & NeeleySIGGRAPH 92
π’π’ π£π£
πππ
Feature-Based WarpingHow do we calculate π’π’ and π£π£?
Recall:π£π£1, π£π£2 = π₯π₯1 β π₯π₯2 + π¦π¦1 β π¦π¦2
= π£π£1 β π£π£2 β cosππ
The (signed) length of theprojection of π£π£1 on the linethrough π£π£2, scaled by thelength of π£π£2.
β The signed length of the projectionof π£π£1 onto the line through π£π£2 is:
π£π£1, π£π£2π£π£2
π£π£1 = (π₯π₯1,π¦π¦1)
π£π£2 = π₯π₯2,π¦π¦2
ππ
How do we calculate π’π’ and π£π£?
π£π£1, π£π£2 is the (signed) length of the projection of π£π£1on the line through π£π£2, scaled by the length of π£π£2.
π£π£ =ππ β π π , π‘π‘ β π π β₯
π‘π‘ β π π β₯
Feature-Based Warping
π£π£1 π£π£2
ππ
π£π£π‘π‘
π π π£π£
How do we calculate π’π’ and π£π£?
π£π£1, π£π£2 is the (signed) length of the projection of π£π£1on the line through π£π£2, scaled by the length of π£π£2.
π’π’ =ππ β π π , π‘π‘ β π π
π‘π‘ β π π β
1π‘π‘ β π π
Feature-Based Warping
π£π£1 π£π£2
Remember: π’π’ is a fraction
ππ
π’π’
π‘π‘
π π
Feature-Based Warpingβ’ [Beier & Neeley, 1992] use a pair of lines to specify
the warp Given ππ in destination image, where is πππ in the source? Describe ππ relative to the destination line Map the description to the source
Beier & NeeleySIGGRAPH 92
Source image Destination image
ππ
π’π’
π£π£
π’π’ is a signed fractionπ£π£ is a signed length (in pixels)
π’π’ π£π£
πππ
Warping with One Line Pairβ’ What happens to the βFβ?
Translation!
Warping with One Line Pairβ’ What happens to the βFβ?
Non-uniform scale!
Warping with One Line Pairβ’ What happens to the βFβ?
Rotation!
Warping with One Line Pairβ’ What happens to the βFβ?
What types of affine transformations canβt be specified?
Warping with One Line Pairβ’ Canβt specify arbitrary scales, skews, mirrors,
angular changesβ¦
X
Warping with Multiple Line Pairsβ’ Use weighted combination of points defined by
each pair of corresponding lines
Beier & Neeley, Figure 4
Warping with Multiple Line Pairsβ’ Use weighted combination of points defined by
each pair of corresponding lines
Source image Destination image
ππMapping
ππ2ππΏπΏ1π πΏπΏ2π πΏπΏ1
πΏπΏ2
ππ1π
Warping with Multiple Line Pairsβ’ Use weighted combination of points defined by
each pair of corresponding lines
Source image Destination image
ππMapping
πππ
πππ is a weighted average
πΏπΏ1π πΏπΏ2π πΏπΏ1πΏπΏ2
Weighting Effect of Each Line Pairβ’ Given a set of line pairs {πΏπΏππππ 0 , β¦ , πΏπΏππππ ππ } and
{πΏπΏππππππ 0 , β¦ , πΏπΏππππππ ππ }, to weight the contribution of each line pair, [Beier & Neeley, 1992] use:
weight ππ ππ ~length ππ ππ
ππ + dist ππ ππ
ππ
where:β’ length[ππ] is the length of line πΏπΏππππππ[ππ]β’ dist[ππ](ππ) is the distance from ππ to πΏπΏππππππ[ππ]β’ ππ (small), ππ β [0.5,2.0], ππ β [0.0,1.0] are
constants that control the warp
How do we calculate dist?
dist ππ = οΏ½π£π£ if π’π’ β [0,1]ππ β π π if π’π’ < 0ππ β π‘π‘ if π’π’ > 1
Feature-Based Warping
ππ
π’π’
π£π£π‘π‘
π π
Given a source image and set of corresponding line segment pairs:
β’ Iterate over each pixel in the target For each pair of line segments
Β» Compute the corresponding position in the sourceΒ» Compute the weights
Average to get the final source position Sample the source (at the source position) to get the
color at the target pixel
Warping
Warping PseudocodeWarp( Imgin , Lin[β¦] , Lout[β¦] ){
foreach destination pixel pout:pin = (0,0)sum = 0for i = 1 to number of line pairs:
qin = pout transformed by ( Lin[i] , Lout[i] )pin += qin * weight[i]( pout ) sum += weight[i]( pout )
pin /= sumImgout(pout) = Imgin( pin )
return Imgout}
Given two images, given a set of corresponding line segment pairs, and an interpolation time πΌπΌ β [0,1]:
β’ Compute the πΌπΌ-blend of the of line segments (by blending the end-points)
β’ Warp the first image using the first set of line segments and the blended line segments
β’ Warp the second image using the second set of line segments and the blended line segments
β’ Compute the πΌπΌ-blend of the warped images
Morphing at πΆπΆ β [ππ,ππ]
Morphing at πΆπΆ β [ππ,ππ] PseudocodeMorph( Img0 , L0[N] , Img1 , L1[N] , πΌπΌ ){
foreach i β {1,β¦,N}:L[i] = line πΌπΌ-th of the way from L0[i] to L1[i]
Warp0 = Warp( Img0 , L0[] , L[] )Warp1 = Warp( Img1 , L1[] , L[] )
return (1-πΌπΌ)*Warp0 + πΌπΌ*Warp1}
warp
πΌπΌ-blend
Animation PseudocodeAnimate( Img0 , L0[N] , Img1 , L1[N], Imgsout[T+1] ){
foreach tβ{0,β¦,T}:Imgsout[t] = Morph(Img0 , L0[N] , Img1 , L1[N] , t/T )
}
MorphingCheck out Michael Jacksonβs βBlack or Whiteβ video at:
https://www.youtube.com/watch?v=pTFE8cirkdQ
[Beier & Neeley, 1992] Example
Image0
Image1
Warp0
Warp1
Result
[Beier & Neeley, 1992] Example
Img0
Img1
Warp0
Warp1
Result
[Beier & Neeley, 1992] Example
Warp0
Warp1
Result
Img0
Img1
Extensions:
β’ Apply to consecutive frames in a video to generate in-between frames (for slow-motion).
β’ Automate the calculation of correspondences.
Limitations:
β’ Occlusions
[Beier & Neeley, 1992] Example
Image Processing
β’ Quantization Uniform quantization Random dither Ordered dither Floyd-Steinberg dither
β’ Pixel operations Add random noise Compute luminance Change contrast Change saturation
β’ Filtering Blur Detect edges
β’ Sampling Nyquist rate Aliasing Ideal filter
β’ Morping Blending Warp