aam based face tracking with temporal matching and face segmentation
DESCRIPTION
CVPR 2010. AAM based Face Tracking with Temporal Matching and Face Segmentation. Mingcai Zhou 1 、 Lin Liang 2 、 Jian Sun 2 、 Yangsheng Wang 1. 1 Institute of Automation Chinese Academy of Sciences, Beijing, China. 2 Microsoft Research Asia Beijing, China. Problems- AAM tracker. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/1.jpg)
1
AAM based Face Tracking with Temporal Matching and Face
Segmentation
1Institute of Automation Chinese Academy of Sciences, Beijing, China2Microsoft Research Asia Beijing, China
Mingcai Zhou1 、 Lin Liang2 、 Jian Sun2 、 Yangsheng Wang1
CVPR 2010
![Page 2: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/2.jpg)
2
![Page 3: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/3.jpg)
3
Problems- AAM tracker
• Difficultly generalize to unseen images• Clutterd backgrounds
![Page 4: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/4.jpg)
4
How to do?
• A temporal matching constraint in AAM fitting- Enforce an inter-frame local appearance constraint between
frames
• Introduce color-based face segmentation as a soft constraint
![Page 5: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/5.jpg)
5
Temporal Matching Constraint
1. Select feature points with salient local appearances at previous frame2. I(t−1) to the Model coordinate and get the appearance A(t-1)
3. Use warping function W(x;pt) maps R(t-1) to a patch R(t) at frame t
![Page 6: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/6.jpg)
6
Shape Initialization
Those feature points whose motion directions are inconsistent with themain direction are most likely to be outliers.
t-1 t
Face Motion Direction
Improve the stability in tracking fast face motions
![Page 7: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/7.jpg)
7
Face Segmentation Constraint
![Page 8: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/8.jpg)
8
Face Segmentation Constraint
![Page 9: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/9.jpg)
9
Experiments
![Page 10: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/10.jpg)
10
Image and Video Abstraction byAnisotropic Kuwahara Filtering
1Hasso-Plattner-Institut, Germany2University of Missouri, St. Louis
Jan Eric Kyprianidis1 、 Henry Kang2 、 Jürgen Döllner1
PG 2009
![Page 11: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/11.jpg)
11
Features
preserving shape boundaries exhibit directional information as found in oil paintings use to video without extra processing
![Page 12: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/12.jpg)
12
Edge-Preserving Filter
![Page 13: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/13.jpg)
13
Edge-Preserving Filter
Kuwahara filter [KHEK76] removes detail in high-contrast regions while also protecting shape boundaries in low-contrast regions.
![Page 14: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/14.jpg)
14
Block Artifacts
![Page 15: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/15.jpg)
15
Anisotropic Filter
![Page 16: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/16.jpg)
16
Method
Orientation and Anisotropy Estimation Anisotropic Kuwahara Filter
![Page 17: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/17.jpg)
17
Experiments
![Page 18: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/18.jpg)
18
Fast Motion Deblurring
POSTECH
Sunghyun Cho、 Seungyong Lee
SIGGRAPH Asia 2009
![Page 19: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/19.jpg)
19
Features
fast deblurring method
![Page 20: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/20.jpg)
20
Single Image Blind Deconvolution
![Page 21: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/21.jpg)
21
Prediction
Image gradient mapsBlurred image
1.Suppress noise: bilateral filter2.Restore strong edge: shock filter3.Gradient magnitude threshold
![Page 22: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/22.jpg)
22
Kernel Estimation
Image gradient maps
Minimize the energy function:
![Page 23: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/23.jpg)
23
Kernel Estimation
CG method
A size: (5n^2) x (m^2) , L:n x n ,K:m x m
![Page 24: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/24.jpg)
24
Kernel Estimation
![Page 25: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/25.jpg)
25
Kernel Estimation
![Page 26: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/26.jpg)
26
Deconvolution
latent image
![Page 27: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/27.jpg)
27
Experiments
![Page 28: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/28.jpg)
28
Experiments
![Page 29: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/29.jpg)
29
Noise Brush: Interactive High Quality Image-Noise Separation
Jia Chen1 、 Chi-Keung Tang1 、 Jue Wang2
1The Hong Kong University of Science and Technology2Adobe Systems, Inc.
SIGGRAPH Asia 2009
![Page 30: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/30.jpg)
30
Problems-denoising
• Over-smoothed image structure• Residual noise in smooth regions
![Page 31: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/31.jpg)
31
Joint Image-Noise Filtering
Purpose:
W大小spatial
color
Image structure
![Page 32: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/32.jpg)
32
Joint Image-Noise Filtering
![Page 33: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/33.jpg)
33
Result
![Page 34: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/34.jpg)
34
Noisy InputJoint Flash Nonflash ResultSingle Image Denoised by NoisewareSingle Image Our Result
![Page 35: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/35.jpg)
35
Result
![Page 36: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/36.jpg)
36
Inverse Texture Synthesis
Li-Yi Wei1 、 Jianwei Han2 、 Kun Zhou1,2
Hujun Bao2 、 Baining Guo1 、 Harry Shum1
1Microsoft Research Asia 2Zhejiang University
SIGGRAPH 2008
![Page 37: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/37.jpg)
37
Flow Diagram
New
![Page 38: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/38.jpg)
38
What is the mean of Inverse?
input
output
inverse texture synthesis
• From a large input texture– produce a small output that best summarizes input
http://www.youtube.com/watch?v=w5HY2xMCldI
![Page 39: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/39.jpg)
39
Benefits
• Reduce storage size• Increase processing speed
890^2X11 128^2X11
![Page 40: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/40.jpg)
40
Generate Compaction
For globally varying texture:
![Page 41: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/41.jpg)
41
Generate Compaction
![Page 42: AAM based Face Tracking with Temporal Matching and Face Segmentation](https://reader035.vdocuments.net/reader035/viewer/2022062809/5681577a550346895dc51542/html5/thumbnails/42.jpg)
42
Generate Compaction