幻灯片 1shiboxin/files/shi_icme09...key laboratory of machine perception (ministry of education)...
TRANSCRIPT
![Page 1: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/1.jpg)
ICME2009BLOCK-BASED COLOR CORRECTION ALGORITHM
FOR MULTI-VIEW VIDEO CODING
Boxin Shi, Yangxi Li, Lin Liu, Chao XuKey Laboratory of Machine Perception (Ministry of Education)Peking University, Beijing, China
Lectured by: Boxin Shi ([email protected])
![Page 2: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/2.jpg)
Outline
1
Problem
MVC
Color Correction
Existed works
2
Algorithm
3
Result
Workflow
Colorization
Local-to-global
Picture
Histogram
Table
![Page 3: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/3.jpg)
Outline
1
Problem
MVC
Color Correction
Existed works
2
Algorithm
3
Result
Workflow
Colorization
Local-to-global
Picture
Histogram
Table
![Page 4: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/4.jpg)
1. Problem
Multi-view Video Coding (MVC)
[1] W. Matusik, H. Pfister. “3DTV: A scalable system for real-time acquisition,
transmission, and autostereoscopic display of dynamic scenes”, ACM
Transactions on Graphics, 2004, 23(3): 814-824.
[2] Tanimoto Lab, Nagoya University. http://www.tanimoto.nuee.nagoya-u.ac.jp.
[1]
[2]
• New experience in video
• Challenge: huge data
• Similarity in different views
• Temporal VS. spatial motion
compensation
![Page 5: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/5.jpg)
1. Problem
Color variation among different views
[3] ISO/IEC JTC1/SC29/WG11, “KDDI Multi-View Video Sequences for MPEG
3DAV Use”, MPEG2004/M10533, Munich, Mar. 2004.
[1]
[2]
• Unavoidable cause
scene illumination
camera calibration
jitter speed
• Bad effects to
visual quality
coding efficiency
[3]
![Page 6: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/6.jpg)
[4] U. Fecker, M. Barkowsky, A. Kaup, “Histogram-Based Prefiltering for Luminance
and Chrominance Compensation of Multiview Video”, IEEE Trans. Circuits & Systems
for Video Technology, vol. 18(9), pp. 1258 – 1267, Sep. 2008.
[5] Y. Chen, C. Cai, J. Liu, “YUV Correction for Multi-View Video Compression”,
ICPR2006, Hong Kong, pp. 734 – 737, Aug 2006.
1. Problem
Existed color correction methods
[1]
[2]
[3]
![Page 7: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/7.jpg)
Outline
1
Problem
MVC
Color Correction
Existed works
2
Algorithm
3
Result
Workflow
Colorization
Local-to-global
Picture
Histogram
Table
![Page 8: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/8.jpg)
2. Algorithm
Overall workflow
[1]
[2]
[3]
![Page 9: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/9.jpg)
2. Algorithm
Colorization scheme
[1]
[2]
[3]• Scribble-based
color blocks as scribbles
• Y guides U/V
similar intensity ~ similar color
• Optimization [6]
[6] A. Levin, D. Lischinski, Y. Weiss, “Colorization
using Optimization”, ACM Trans. Graph., vol. 23(3),
pp. 689 – 694, Aug. 2004.
![Page 10: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/10.jpg)
2. Algorithm
Global mixture
[1]
[2]
[3]• Block scribble shortcomings
square is not enough
color bleeding
• Global refinement
detect wrongly colored pixels
color transfer [7]
lαβ color space
[7] E. Reinhard, M. Ashikhmin, B. Gooch, and P.
Shirley, Color transfer between images, IEEE
Computer Graphics Applications, vol. 21(5), pp.
34 – 41, 2001.
![Page 11: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/11.jpg)
Outline
1
Problem
MVC
Color Correction
Existed works
2
Algorithm
3
Result
Workflow
Colorization
Local-to-global
Picture
Histogram
Table
![Page 12: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/12.jpg)
3. Result
Picture (Visual quality comparison)
[1]
[2]
[3]
• Unavoidable cause:
scene illumination
camera calibration
jitter speed
• Bad effects to:
visual quality
coding efficiency
![Page 13: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/13.jpg)
3. Result
Histogram (YUV space)
[1]
[2]
[3]
![Page 14: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/14.jpg)
3. Result
Table (ratio of best matching blocks in spatial prediction)
[1]
[2]
[3]• Spatial motion compensation
• Ratio = No. of best matching blocks / total No.
• Partially reflect the correction effectiveness
• PSNR can be influence by various factors
![Page 15: 幻灯片 1shiboxin/files/Shi_ICME09...Key Laboratory of Machine Perception (Ministry of Education) Peking University, Beijing, China Lectured by: Boxin Shi (shiboxin@cis.pku.edu.cn)](https://reader035.vdocuments.net/reader035/viewer/2022071610/614913749241b00fbd675336/html5/thumbnails/15.jpg)