3d stereo video coding heejune ahn embedded communications laboratory seoul national univ. of...
TRANSCRIPT
3D Stereo Video Coding
Heejune AHNEmbedded Communications Laboratory
Seoul National Univ. of TechnologyFall 2013
Last updated 2013. 9. 31
Heejune AHN: Image and Video Compression p. 2
Outline
Stereo video display Stereo video principle Multi-View Coding
H.264 amendment HEVC extension
Heejune AHN: Image and Video Compression p. 3
3D display
Stereo (existing) Anaglyphic (passive red-cyan glasses) Polarized (passive polarized glasses) Shutter/Synchronized (alternative frame sequencing) Auto stereographic (No glasses)
Hologram
Heejune AHN: Image and Video Compression p. 5
Stereo Display
Polarization type micro-polarizer
(patterned retarder)
polarized glass
Active retarder
½ ver. resol.
full ver. resol.
Heejune AHN: Image and Video Compression p. 6
Active Shutter glasses type
2 times frame rate (i.e. 120 Hz)
synchronization
• cross-talk due to incomplete sync.• eye fatigue• more expensive glasses than polarized
L R
Heejune AHN: Image and Video Compression p. 7
No glasses Lenti-cular - Array of
lenslets in front of the pixel raster; relies on refraction of light through lenslets.
Parallax barrier - A barrier mask is placed in front of the pixel raster to control the light emitted to certain viewing positions
Limitation on distance
Heejune AHN: Image and Video Compression p. 10
The Principle of Stereo Vision
Disparity between two eyes
Heejune AHN: Image and Video Compression p. 13
Stereo Video Coding
Frame Compatible Stereo Spatial: side-by-side, top-bottom, line/row interleaved Time: views interleaved as alternating frames or fields +: Leverage existing infrastructure (2D coding) - : Reduced-resolution, coding in-efficiency Currently in TV broadcast service
Multiview Video Coding (MVC) H.264 amendment (2011)
• 3D Video and Free Viewpoint Video (FVV) Exploits correlation among views + : Full-resolution Adopted as format for 3D Blue-ray Disc
Heejune AHN: Image and Video Compression p. 14
Frame-compatible Stereo Video Top-bottom, side-by-side, line-row interleaved Example
Heejune AHN: Image and Video Compression p. 15
H.264/MVC
statistical dependence in temporal and inter-views temporal, spatial, plus inter-view prediction Backwards compatible to H.264 single view profiles
Heejune AHN: Image and Video Compression p. 16
MVD Coding
Multi-View(2D)+Depth Coding HEVC extension Better for auto-stereoscopic with DIBR (Depth Information Based
Rendering)
Depth maps : distance from the cameras/view points • inverse proportional to disparity
Heejune AHN: Image and Video Compression p. 17
Block diagram MV capturing, coding, decoding, and rendering
Heejune AHN: Image and Video Compression p. 18
Coding algorithm 1 base + N-1 dependent views N associate depth maps
Heejune AHN: Image and Video Compression p. 20
Interview residual signal prediction Predict ‘residual’ of a current block using a coded residual block in a reference view Determined disparity using the estimated depth maps (same depth as for
inter-view motion parameter prediction) Disparity-compensated prediction of the current residual Bilinear filtering
for sub-sample interpolation use after motion compensation
residual signal depthmap
Heejune AHN: Image and Video Compression p. 21
Depth map coding
Depth map properties1. Sharp edges representing object
borders
2. Large areas of slowly varying values representing object areas
3. Edges in depth maps are correlated with edges in video pictures
Depth map coding ideas 1. Representation of depth edges
2. Partition block into two regions with constant sample values
3. Prediction based on co-located texture block
4. Optional transform coding of residual
Heejune AHN: Image and Video Compression p. 22
Non-rectangular block partitions Approximating the signal of a depth block by a model that partitions
the area into two non-rectangular regions Each region is represented by a constant value