![Page 1: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/1.jpg)
An Interactive Background Blurring Mechanism and Its Applications
NTU CSIE
Yan Chih-Yu
Advisor: Wu Ja-Ling, Ph.D.
![Page 2: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/2.jpg)
2
![Page 3: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/3.jpg)
3
Outline IntroductionRelated WorkMethod
◦Object Segmentation◦Depth Map Generation◦Image Defocus
Experimental ResultApplicationsConclusion
![Page 4: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/4.jpg)
Introduction
Real world
Photo plane
Depth of field◦Circle of confusion
![Page 5: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/5.jpg)
5
Photo plane
Introduction
Real light
Circle of Confusion
Out of focus
Depth of field◦Circle of confusion
![Page 6: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/6.jpg)
Introduction Depth of field
Real light
Circle of Confusion- Readability range
Focus plane Photo planeOut of focus – blur range
Depth of field
![Page 7: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/7.jpg)
Introduction
Real world
Photo plane
Shallow focus
![Page 8: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/8.jpg)
8
IntroductionShallow focus
◦Highlight the subject by softening background diffusion
Deep focus by DC
NIKON E4300 (2003)
Shallow focus by DSLR
NIKON D90(2008)
![Page 9: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/9.jpg)
Method
Lazy SnappingLazy Snapping
Alpha MattingAlpha Matting
Face DetectionFace Detection
Perspective BoxPerspective Box
Pop-up CardPop-up Card
Camera SettingsCamera Settings
Defocus BlurDefocus Blur
Privacy PreservingPrivacy Preserving
Photo BrowserPhoto Browser
Object Segmentation
Depth Map Generation
Applications
Image Defocus
![Page 10: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/10.jpg)
10
Method
Segment◦Lazy snapping◦Mean shift◦Alpha matting◦Face detection
Resize to 20%
Do hard graph cut by lazy snapping
User Stroke
Mean shift
Do hard graph cut by lazy snapping
User Stroke
Resize to 20%
Mean shift
Segmentation tri-Map
Segmentation alpha Map
Lazy Snapping. ACM Trans. On Graphics 2004.Mean Shift: A Robust Approach Toward Feature Space Analysis. PAMI 2002.A Bayesian Approach to Digital Matting. CVPR 2001.
![Page 11: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/11.jpg)
11
MethodDepth map
◦ Perspective Box Vanish point Rear wall
Depth Map
Perspective Box
Normal vector
Pop-up card
Depth Map
Perspective Box
Normal vector
Pop-up card
Tour Into the Picture: Using a spidery mesh user interface to make animation from a single image. SIGGRAPH 1997
![Page 12: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/12.jpg)
12
MethodDepth map
◦ Pop-up card
Depth Map
Perspective Box
Normal vector
Pop-up card
Depth Map
Normal vector
Pop-up card
![Page 13: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/13.jpg)
13
MethodDepth map
◦ Normal vector
Depth Map
Perspective Box
Normal vector
Pop-up card
Depth Map
Normal vector
![Page 14: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/14.jpg)
14
Method
Image defocus◦Blur circle diameter
Blur circle diameter
Segment Map
Depth Map
Camera setting
Aperture sizeFocal length of the
lensDistance of focus
u v
bd
vbub
u v
bd
vbub
(a) (b)𝑏= 𝑓𝑁ȁ�(𝑢𝑏 − 𝑢)𝑓ȁ�𝑢𝑏(𝑢− 𝑓) 1𝑢+ 1𝑣 = 1𝑓
![Page 15: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/15.jpg)
15
MethodDefocus blur
◦bokeh
𝐼𝑑𝑒𝑓𝑜𝑐𝑢𝑠 ሺ𝑖,𝑗ሻ= σ𝐸𝑛ሺ𝑖,𝑗ሻσ𝑤𝑒𝑖𝑔ℎ𝑡𝑛(𝑖,𝑗) + 𝛼𝑛(𝑖,𝑗)
𝐸𝑛ሺ𝑖,𝑗ሻ= 𝐼ሺ𝑥𝑛,𝑦𝑛ሻ 𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛,𝑠,𝑏)Τ
𝑤𝑒𝑖𝑔ℎ𝑡𝑛ሺ𝑖,𝑗ሻ= 1 𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛,𝑠,𝑏)Τ
𝛼𝑛(𝑖,𝑗) =൜𝑤𝑒𝑖𝑔ℎ𝑡𝑛(𝑖,𝑗), 𝐼ሺ𝑥𝑛,𝑦𝑛ሻ< 𝑇ℎ𝑏𝑜𝑘𝑒ℎ0, 𝐼ሺ𝑥𝑛,𝑦𝑛ሻ≥ 𝑇ℎ𝑏𝑜𝑘𝑒ℎ
𝑤𝑒𝑖𝑔ℎ𝑡𝑛ሺ𝑖,𝑗ሻ= 1𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛ሺ𝑥𝑛,𝑦𝑛,𝑠,𝑏ሻ× 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛)𝑙𝑢𝑚𝑖𝑛𝑜𝑢𝑠(𝑥𝑛,𝑦𝑛)
……
……E1(i,j)
I(x1,y1) I(x2,y2) I(x3,y3)
I(xn,yn) I(xn+1,yn+1)
E2(i,j) E3(i,j)
En(i,j) En+1(i,j)
En+2(i,j)
I(xn+2,yn+2)
![Page 16: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/16.jpg)
16
MethodDefocus blur
……
……E1(i,j)
I(x1,y1) I(x2,y2) I(x3,y3)
I(xn,yn) I(xn+1,yn+1)
E2(i,j) E3(i,j)
En(i,j) En+1(i,j)
En+2(i,j)
I(xn+2,yn+2)
![Page 17: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/17.jpg)
17
Method
◦Near-by object case
Focus on the flower
Focus on the flower
![Page 18: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/18.jpg)
18
◦Near-by object case
Method Blur circle diameter
Objects in front of the
focus
Objects behind
the focus
Defocus Blur
Text synthesis inpainting
Blur shape as alpha map
Defocus Blur
Interpolation by alpha map
Shallow focus image
![Page 19: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/19.jpg)
19
Experimental Result 1Defocus blur method proposed in the system
comparing with other blur filter results
Deep focus Shallow focus
![Page 20: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/20.jpg)
20
Experimental Result 1Defocus blur method proposed in the system
comparing with other blur filter results
Gaussian blur Defocus blur Defocus blur + Bokeh
![Page 21: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/21.jpg)
21
Experimental Result 1
![Page 22: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/22.jpg)
22Defocus blur
Experimental Result 1
Gaussian blur Defocus blur + Bokeh
Original photograph
• Defocus blur method proposed in the system comparing with other blur filter results
![Page 23: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/23.jpg)
23
Experimental Result 2with / without depth variation in the
background
Real photograph taken by DSLR
![Page 24: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/24.jpg)
24
Experimental Result 2with / without depth
variation in the background
Original photograph
Result after post-processing
Depth variation
Depth fixed
![Page 25: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/25.jpg)
25
Experimental Result 3
◦Near-by object case
Focus on the flower
Focus on the flower
![Page 26: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/26.jpg)
26
◦Near-by object case
Without inpainting
Interpolation result
Experimental Result 3
![Page 27: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/27.jpg)
27
Experimental Result 4Comparing with Photoshop
Photoshop (1 Hour) Our system (3~5min)
Original image
![Page 28: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/28.jpg)
28
Applications - Privacy Preserving
![Page 29: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/29.jpg)
29
Applications - Partial Viewing
![Page 30: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/30.jpg)
30
Applications - Partial Viewing
![Page 31: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/31.jpg)
31
Applications - Image Transition at Photo Browser
![Page 32: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/32.jpg)
32
Applications - Image Transition at Photo Browser
![Page 33: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/33.jpg)
33
Movie with defocus blur
effect
Applications - Image Transition at Photo Browser
![Page 34: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/34.jpg)
34
Applications - Image Transition at Photo Browser
Concatenate two unrelated
images
![Page 35: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/35.jpg)
35
ConclusionWe proposed an interactive refocusing
tool for background blurring◦ Simple user hint◦ Defocus blur kernel ◦ Concatenate two related picture
Future work◦ concatenate two or more unrelated
pictures Color based image retrieval technique
![Page 36: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/36.jpg)
36
Q&A
![Page 37: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/37.jpg)
37
Experimental Result 1Defocus blur method proposed in the system
comparing with other blur filter results
Deep focus Shallow focus Difference
![Page 38: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/38.jpg)
38
Experimental Result 1Defocus blur method proposed in the system
comparing with other blur filter results
Shallow focus
Difference
Defocus blurPSNR:22.79
Gaussian blurPSNR:20.34
![Page 39: An Interactive Background Blurring Mechanism and Its Applications](https://reader035.vdocuments.net/reader035/viewer/2022062322/568130b3550346895d96cb73/html5/thumbnails/39.jpg)
39
END
THANK YOU