Slide 1

Video Matting

Ehsan Shahrian, Brian Price

1

1

2Video Matting- Difference in input

2

3Video Matting- Difference in input

3

4Video Matting- Coherence

4

5Video Matting- IntroductionMost video methods are extensions of image methodsCommon extensionsPropagate/generate trimapCompute matte using temporal informationRefine matte using temporal information

5

6Video Matting- Realtime Video MattingGong et al. Realtime video matting using multichannel Poisson equations. Graphics Interface 2010.

Video InputAutomatic binary segmentation and boundary dilationSingle-frame image mattingResult

6

7Video Matting- Video matting via opacity propagationTang et al. Video matting via opacity propagation. Visual Computer, 2012.Tang et al. Temporally consistent video matting based on bilayer segmentation. ICME 2010.

7

8Video Matting- Video matting via opacity propagationTang et al. Video matting via opacity propagation. Visual Computer, 2012.Tang et al. Temporally consistent video matting based on bilayer segmentation. ICME 2010.3D matting Laplacian

8

9Video Matting- Bayesian Video MattingApostoloff et al. Bayesian video matting using Learnt Image Priors. CVPR 2004.

9

10Video Matting- Bayesian Video MattingApostoloff et al. Bayesian video matting using Learnt Image Priors. CVPR 2004.

10

11Video Matting- Video Matting of Complex ScenesChuang et al. Video Matting of Complex Scenes. TOG 2002.

Input trimap and garbage matte at a number of frames

Trimap interpolation using optical flowBackground estimation using background mosaicing

Bayesian matting single-frame, using bg estimation at some pixels

11

12Video Matting- Multiframe nonlocal matting LaplacianChoi et al. Video matting using multiframe nonlocal matting Laplacian. ECCV 2012.

12

Spatiotemporal RefinementNonlocal mean regularization13Video Matting- Multiframe nonlocal matting LaplacianChoi et al. Video matting using multiframe nonlocal matting Laplacian. ECCV 2012.

13

14Video Matting- Motion-Aware KNN LaplacianLi et al. Motion-aware KNN Laplacian for video matting. ICCV 2013.

14

15Video Matting- Temporally Coherent Video MattingLee et al. Temporally coherent video matting. Graphical Models 2010.

15

16Video Matting- Video Snapcut

Temporal coherency Color coherenceSmoothness

16

17

Video MattingToward Temporally Coherent Video MattingX. Bai, J. Wang, and D. Simons, Towards temporally-coherent video matting, Computer Vision/Computer Graphics Collaboration Techniques, pp. 63-74, 2011

17

18

Video Matting- Temporally Coherent and Spatially Accurate

18

19

Sampling strategyVideo Matting- Temporally Coherent and Spatially Accurate

19

20

Sampling strategyVideo Matting- Temporally Coherent and Spatially Accurate

20

21

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, High accuracy optical flow estimation based on a theory for warping," pp. 25-36, 2004.

Frame tEstimated matteFrame t+1Warped matteSelection of best (F,B) pairVideo Matting- Temporally Coherent and Spatially Accurate

21

22

Temporal LaplacianConventional LaplacianFramesPost processingVideo Matting- Temporally Coherent and Spatially Accurate

23

Video Matting- Temporally Coherent and Spatially Accurate

23

24Video Matting- Multiple video streamsMcGuire et al. Defocus Video Matting. Siggraph 2005.

Joshi et al. Natural Video Matting using Camera Arrays. Siggraph 2006.

24

Demo- Adobe AfterEffects Refine Edge

25

26

26