Slide 1

Video Matting

Ehsan Shahrian, Brian Price

112Video Matting- Difference in input

23Video Matting- Difference in input

34Video Matting- Coherence

45Video Matting- IntroductionMost video methods are extensions of image methodsCommon extensionsPropagate/generate trimapCompute matte using temporal informationRefine matte using temporal information

56Video Matting- Realtime Video MattingGong et al. Realtime video matting using multichannel Poisson equations. Graphics Interface 2010.

Video InputAutomatic binary segmentation and boundary dilationSingle-frame image mattingResult67Video Matting- Video matting via opacity propagationTang et al. Video matting via opacity propagation. Visual Computer, 2012.Tang et al. Temporally consistent video matting based on bilayer segmentation. ICME 2010.

78Video Matting- Video matting via opacity propagationTang et al. Video matting via opacity propagation. Visual Computer, 2012.Tang et al. Temporally consistent video matting based on bilayer segmentation. ICME 2010.3D matting Laplacian

89Video Matting- Bayesian Video MattingApostoloff et al. Bayesian video matting using Learnt Image Priors. CVPR 2004.

910Video Matting- Bayesian Video MattingApostoloff et al. Bayesian video matting using Learnt Image Priors. CVPR 2004.

1011Video Matting- Video Matting of Complex ScenesChuang et al. Video Matting of Complex Scenes. TOG 2002.

Input trimap and garbage matte at a number of frames

Trimap interpolation using optical flowBackground estimation using background mosaicing

Bayesian matting single-frame, using bg estimation at some pixels

1112Video Matting- Multiframe nonlocal matting LaplacianChoi et al. Video matting using multiframe nonlocal matting Laplacian. ECCV 2012.

12Spatiotemporal RefinementNonlocal mean regularization13Video Matting- Multiframe nonlocal matting LaplacianChoi et al. Video matting using multiframe nonlocal matting Laplacian. ECCV 2012.

1314Video Matting- Motion-Aware KNN LaplacianLi et al. Motion-aware KNN Laplacian for video matting. ICCV 2013.

1415Video Matting- Temporally Coherent Video MattingLee et al. Temporally coherent video matting. Graphical Models 2010.

15

16Video Matting- Video SnapcutTemporal coherency Color coherenceSmoothness

1617

Video MattingToward Temporally Coherent Video MattingX. Bai, J. Wang, and D. Simons, Towards temporally-coherent video matting, Computer Vision/Computer Graphics Collaboration Techniques, pp. 63-74, 20111718

Video Matting- Temporally Coherent and Spatially Accurate1819

Sampling strategyVideo Matting- Temporally Coherent and Spatially Accurate1920

Sampling strategyVideo Matting- Temporally Coherent and Spatially Accurate2021

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, High accuracy optical flow estimation based on a theory for warping," pp. 25-36, 2004.

Frame tEstimated matteFrame t+1Warped matteSelection of best (F,B) pairVideo Matting- Temporally Coherent and Spatially Accurate2122

Temporal LaplacianConventional LaplacianFramesPost processingVideo Matting- Temporally Coherent and Spatially Accurate23

Video Matting- Temporally Coherent and Spatially Accurate2324Video Matting- Multiple video streamsMcGuire et al. Defocus Video Matting. Siggraph 2005.

Joshi et al. Natural Video Matting using Camera Arrays. Siggraph 2006.

24Demo- Adobe AfterEffects Refine Edge2526

26